2026-02-26T02:53:10.349 INFO:root:teuthology version: 1.2.4.dev11+g6e3d0419c 2026-02-26T02:53:10.363 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-26T02:53:10.442 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-26_01:51:16-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/71112 branch: wip-yuri9-testing-2026-02-25-1600-squid description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '71112' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-26_01:51:16-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri9-testing-2026-02-25-1600-squid ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 workunit: branch: wip-yuri9-testing-2026-02-25-1600-squid sha1: 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1525 sha1: 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri9-testing-2026-02-25-1600-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 targets: trial123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDIE36drgVdSY7Bk3ZqfvAEX4Ooo0dGN05uNp3jbTdIkE5iHYQ2Ojrp+Ysw2rWhLEhFfuuAtksxtuMNOa4NU64g= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - osd teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6e3d0419ccca0e75a03e08feb708946e198236f4 timestamp: 2026-02-26_01:51:16 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-26T02:53:10.442 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/qa; will attempt to use it 2026-02-26T02:53:10.442 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/qa/tasks 2026-02-26T02:53:10.442 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-26T02:53:10.447 INFO:teuthology.task.internal:Checking packages... 2026-02-26T02:53:10.447 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5' 2026-02-26T02:53:10.447 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-26T02:53:10.447 INFO:teuthology.packaging:ref: None 2026-02-26T02:53:10.447 INFO:teuthology.packaging:tag: None 2026-02-26T02:53:10.447 INFO:teuthology.packaging:branch: wip-yuri9-testing-2026-02-25-1600-squid 2026-02-26T02:53:10.447 INFO:teuthology.packaging:sha1: 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 2026-02-26T02:53:10.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri9-testing-2026-02-25-1600-squid 2026-02-26T02:53:10.566 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-687-g8d724284-1jammy 2026-02-26T02:53:10.568 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-26T02:53:10.580 INFO:teuthology.task.internal:no buildpackages task found 2026-02-26T02:53:10.580 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-26T02:53:10.593 INFO:teuthology.task.internal:Saving configuration 2026-02-26T02:53:10.598 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-26T02:53:10.609 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-26T02:53:10.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial123.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-26_01:51:16-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/71112', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-26 02:49:40.352729', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKtQ7P8p+sVvSJKwN1vnZ9XfpW9kXxCUTvP1gLRednJc'} 2026-02-26T02:53:10.695 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-26T02:53:10.707 INFO:teuthology.task.internal:roles: ubuntu@trial123.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-26T02:53:10.707 INFO:teuthology.run_tasks:Running task console_log... 2026-02-26T02:53:10.787 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x714f90856f20>, signals=[15]) 2026-02-26T02:53:10.787 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-26T02:53:10.801 INFO:teuthology.task.internal:Opening connections... 2026-02-26T02:53:10.802 DEBUG:teuthology.task.internal:connecting to ubuntu@trial123.front.sepia.ceph.com 2026-02-26T02:53:10.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T02:53:10.866 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-26T02:53:10.878 DEBUG:teuthology.orchestra.run.trial123:> uname -m 2026-02-26T02:53:10.947 INFO:teuthology.orchestra.run.trial123.stdout:x86_64 2026-02-26T02:53:10.948 DEBUG:teuthology.orchestra.run.trial123:> cat /etc/os-release 2026-02-26T02:53:10.994 INFO:teuthology.orchestra.run.trial123.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-26T02:53:10.994 INFO:teuthology.orchestra.run.trial123.stdout:NAME="Ubuntu" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:VERSION_ID="22.04" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:VERSION_CODENAME=jammy 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:ID=ubuntu 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:ID_LIKE=debian 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-26T02:53:10.995 INFO:teuthology.orchestra.run.trial123.stdout:UBUNTU_CODENAME=jammy 2026-02-26T02:53:10.995 INFO:teuthology.lock.ops:Updating trial123.front.sepia.ceph.com on lock server 2026-02-26T02:53:11.070 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-26T02:53:11.083 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-26T02:53:11.096 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-26T02:53:11.097 DEBUG:teuthology.orchestra.run.trial123:> test '!' -e /home/ubuntu/cephtest 2026-02-26T02:53:11.100 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-26T02:53:11.114 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-26T02:53:11.115 DEBUG:teuthology.orchestra.run.trial123:> test -z $(ls -A /var/lib/ceph) 2026-02-26T02:53:11.152 INFO:teuthology.orchestra.run.trial123.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-26T02:53:11.153 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-26T02:53:11.268 INFO:teuthology.run_tasks:Running task kernel... 2026-02-26T02:53:11.288 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-26T02:53:11.289 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-26T02:53:11.289 DEBUG:teuthology.orchestra.run.trial123:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-26T02:53:11.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T02:53:11.292 DEBUG:teuthology.orchestra.run.trial123:> uname -r 2026-02-26T02:53:11.339 INFO:teuthology.orchestra.run.trial123.stdout:5.15.0-170-generic 2026-02-26T02:53:11.339 INFO:teuthology.task.kernel:Running kernel on trial123: 5.15.0-170-generic 2026-02-26T02:53:11.339 DEBUG:teuthology.orchestra.run.trial123:> sudo apt-get clean 2026-02-26T02:53:11.398 DEBUG:teuthology.orchestra.run.trial123:> sudo apt-get update 2026-02-26T02:53:11.903 INFO:teuthology.orchestra.run.trial123.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-26T02:53:11.996 INFO:teuthology.orchestra.run.trial123.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-26T02:53:12.345 INFO:teuthology.orchestra.run.trial123.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-26T02:53:12.501 INFO:teuthology.orchestra.run.trial123.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-26T02:53:12.646 INFO:teuthology.orchestra.run.trial123.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3274 kB] 2026-02-26T02:53:13.079 INFO:teuthology.orchestra.run.trial123.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [498 kB] 2026-02-26T02:53:13.092 INFO:teuthology.orchestra.run.trial123.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.2 kB] 2026-02-26T02:53:13.092 INFO:teuthology.orchestra.run.trial123.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-26T02:53:13.132 INFO:teuthology.orchestra.run.trial123.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-26T02:53:13.139 INFO:teuthology.orchestra.run.trial123.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-26T02:53:13.140 INFO:teuthology.orchestra.run.trial123.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5400 kB] 2026-02-26T02:53:13.310 INFO:teuthology.orchestra.run.trial123.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1025 kB] 2026-02-26T02:53:13.334 INFO:teuthology.orchestra.run.trial123.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [3007 kB] 2026-02-26T02:53:13.425 INFO:teuthology.orchestra.run.trial123.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [430 kB] 2026-02-26T02:53:13.432 INFO:teuthology.orchestra.run.trial123.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-26T02:53:13.433 INFO:teuthology.orchestra.run.trial123.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1019 kB] 2026-02-26T02:53:13.469 INFO:teuthology.orchestra.run.trial123.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [225 kB] 2026-02-26T02:53:13.478 INFO:teuthology.orchestra.run.trial123.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.6 kB] 2026-02-26T02:53:13.478 INFO:teuthology.orchestra.run.trial123.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5217 kB] 2026-02-26T02:53:13.630 INFO:teuthology.orchestra.run.trial123.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [993 kB] 2026-02-26T02:53:17.065 INFO:teuthology.orchestra.run.trial123.stdout:Fetched 23.1 MB in 3s (7640 kB/s) 2026-02-26T02:53:17.604 INFO:teuthology.orchestra.run.trial123.stdout:Reading package lists... 2026-02-26T02:53:17.618 DEBUG:teuthology.orchestra.run.trial123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-26T02:53:17.653 INFO:teuthology.orchestra.run.trial123.stdout:Reading package lists... 2026-02-26T02:53:17.763 INFO:teuthology.orchestra.run.trial123.stdout:Building dependency tree... 2026-02-26T02:53:17.764 INFO:teuthology.orchestra.run.trial123.stdout:Reading state information... 2026-02-26T02:53:17.852 INFO:teuthology.orchestra.run.trial123.stdout:The following packages were automatically installed and are no longer required: 2026-02-26T02:53:17.852 INFO:teuthology.orchestra.run.trial123.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-26T02:53:17.852 INFO:teuthology.orchestra.run.trial123.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout: sg3-utils-udev 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout:The following additional packages will be installed: 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout: linux-generic linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout: linux-headers-generic linux-image-5.15.0-171-generic 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout: linux-modules-5.15.0-171-generic linux-modules-extra-5.15.0-171-generic 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout:Suggested packages: 2026-02-26T02:53:17.853 INFO:teuthology.orchestra.run.trial123.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-26T02:53:17.866 INFO:teuthology.orchestra.run.trial123.stdout:The following NEW packages will be installed: 2026-02-26T02:53:17.866 INFO:teuthology.orchestra.run.trial123.stdout: linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-26T02:53:17.866 INFO:teuthology.orchestra.run.trial123.stdout: linux-image-5.15.0-171-generic linux-modules-5.15.0-171-generic 2026-02-26T02:53:17.867 INFO:teuthology.orchestra.run.trial123.stdout: linux-modules-extra-5.15.0-171-generic 2026-02-26T02:53:17.867 INFO:teuthology.orchestra.run.trial123.stdout:The following packages will be upgraded: 2026-02-26T02:53:17.867 INFO:teuthology.orchestra.run.trial123.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-26T02:53:18.220 INFO:teuthology.orchestra.run.trial123.stdout:3 upgraded, 5 newly installed, 0 to remove and 31 not upgraded. 2026-02-26T02:53:18.221 INFO:teuthology.orchestra.run.trial123.stdout:Need to get 113 MB of archives. 2026-02-26T02:53:18.221 INFO:teuthology.orchestra.run.trial123.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-26T02:53:18.221 INFO:teuthology.orchestra.run.trial123.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-171-generic amd64 5.15.0-171.181 [22.7 MB] 2026-02-26T02:53:19.325 INFO:teuthology.orchestra.run.trial123.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-171-generic amd64 5.15.0-171.181 [11.6 MB] 2026-02-26T02:53:19.595 INFO:teuthology.orchestra.run.trial123.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-171-generic amd64 5.15.0-171.181 [63.9 MB] 2026-02-26T02:53:21.095 INFO:teuthology.orchestra.run.trial123.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.171.160 [1658 B] 2026-02-26T02:53:21.095 INFO:teuthology.orchestra.run.trial123.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.171.160 [2486 B] 2026-02-26T02:53:21.095 INFO:teuthology.orchestra.run.trial123.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171 all 5.15.0-171.181 [12.4 MB] 2026-02-26T02:53:21.374 INFO:teuthology.orchestra.run.trial123.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171-generic amd64 5.15.0-171.181 [2849 kB] 2026-02-26T02:53:21.437 INFO:teuthology.orchestra.run.trial123.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.171.160 [2332 B] 2026-02-26T02:53:21.634 INFO:teuthology.orchestra.run.trial123.stdout:Fetched 113 MB in 4s (31.8 MB/s) 2026-02-26T02:53:21.655 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package linux-modules-5.15.0-171-generic. 2026-02-26T02:53:21.828 INFO:teuthology.orchestra.run.trial123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-26T02:53:21.830 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../0-linux-modules-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-26T02:53:21.831 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:22.187 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package linux-image-5.15.0-171-generic. 2026-02-26T02:53:22.196 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../1-linux-image-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-26T02:53:22.213 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:22.256 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-171-generic. 2026-02-26T02:53:22.264 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-26T02:53:22.264 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:23.369 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../3-linux-generic_5.15.0.171.160_amd64.deb ... 2026-02-26T02:53:23.370 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-26T02:53:23.395 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.171.160_amd64.deb ... 2026-02-26T02:53:23.395 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-image-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-26T02:53:23.411 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package linux-headers-5.15.0-171. 2026-02-26T02:53:23.420 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../5-linux-headers-5.15.0-171_5.15.0-171.181_all.deb ... 2026-02-26T02:53:23.420 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-26T02:53:24.783 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package linux-headers-5.15.0-171-generic. 2026-02-26T02:53:24.798 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../6-linux-headers-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-26T02:53:24.798 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:25.387 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.171.160_amd64.deb ... 2026-02-26T02:53:25.388 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking linux-headers-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-26T02:53:25.414 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-26T02:53:25.415 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:25.416 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-headers-generic (5.15.0.171.160) ... 2026-02-26T02:53:25.418 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:26.210 INFO:teuthology.orchestra.run.trial123.stdout:I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-26T02:53:26.210 INFO:teuthology.orchestra.run.trial123.stdout:I: /boot/initrd.img.old is now a symlink to initrd.img-5.15.0-170-generic 2026-02-26T02:53:26.210 INFO:teuthology.orchestra.run.trial123.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-171-generic 2026-02-26T02:53:26.211 INFO:teuthology.orchestra.run.trial123.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-171-generic 2026-02-26T02:53:26.219 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:27.034 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:27.839 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-image-generic (5.15.0.171.160) ... 2026-02-26T02:53:27.840 INFO:teuthology.orchestra.run.trial123.stdout:Setting up linux-generic (5.15.0.171.160) ... 2026-02-26T02:53:27.841 INFO:teuthology.orchestra.run.trial123.stdout:Processing triggers for linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-26T02:53:27.846 INFO:teuthology.orchestra.run.trial123.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-26T02:53:27.846 INFO:teuthology.orchestra.run.trial123.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-171-generic 2026-02-26T02:53:34.974 INFO:teuthology.orchestra.run.trial123.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-26T02:53:41.562 INFO:teuthology.orchestra.run.trial123.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-26T02:53:41.562 INFO:teuthology.orchestra.run.trial123.stdout:Sourcing file `/etc/default/grub' 2026-02-26T02:53:41.581 INFO:teuthology.orchestra.run.trial123.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-26T02:53:41.582 INFO:teuthology.orchestra.run.trial123.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-26T02:53:41.585 INFO:teuthology.orchestra.run.trial123.stdout:Generating grub configuration file ... 2026-02-26T02:53:41.667 INFO:teuthology.orchestra.run.trial123.stdout:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-26T02:53:41.674 INFO:teuthology.orchestra.run.trial123.stdout:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-26T02:53:41.845 INFO:teuthology.orchestra.run.trial123.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-26T02:53:41.851 INFO:teuthology.orchestra.run.trial123.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-26T02:53:41.894 INFO:teuthology.orchestra.run.trial123.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-26T02:53:41.900 INFO:teuthology.orchestra.run.trial123.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-26T02:53:41.974 INFO:teuthology.orchestra.run.trial123.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-26T02:53:41.974 INFO:teuthology.orchestra.run.trial123.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-26T02:53:41.974 INFO:teuthology.orchestra.run.trial123.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-26T02:53:41.977 INFO:teuthology.orchestra.run.trial123.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-26T02:53:41.987 INFO:teuthology.orchestra.run.trial123.stdout:done 2026-02-26T02:53:42.392 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.392 INFO:teuthology.orchestra.run.trial123.stdout:Pending kernel upgrade! 2026-02-26T02:53:42.392 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.392 INFO:teuthology.orchestra.run.trial123.stdout:Running kernel version: 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout: 5.15.0-170-generic 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout:Diagnostics: 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout: The currently running kernel version is not the expected kernel version 5.15.0-171-generic. 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout:The processor microcode seems to be up-to-date. 2026-02-26T02:53:42.393 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout:No services need to be restarted. 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout:No containers need to be restarted. 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout:No user sessions are running outdated binaries. 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:53:42.394 INFO:teuthology.orchestra.run.trial123.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-26T02:53:43.232 DEBUG:teuthology.orchestra.run.trial123:> dpkg -s linux-image-generic 2026-02-26T02:53:43.243 INFO:teuthology.orchestra.run.trial123.stdout:Package: linux-image-generic 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Status: install ok installed 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Priority: optional 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Section: kernel 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Installed-Size: 22 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Maintainer: Ubuntu Kernel Team 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Architecture: amd64 2026-02-26T02:53:43.244 INFO:teuthology.orchestra.run.trial123.stdout:Source: linux-meta 2026-02-26T02:53:43.245 INFO:teuthology.orchestra.run.trial123.stdout:Version: 5.15.0.171.160 2026-02-26T02:53:43.245 INFO:teuthology.orchestra.run.trial123.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-26T02:53:43.245 INFO:teuthology.orchestra.run.trial123.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-26T02:53:43.245 INFO:teuthology.orchestra.run.trial123.stdout:Recommends: thermald 2026-02-26T02:53:43.245 INFO:teuthology.orchestra.run.trial123.stdout:Description: Generic Linux kernel image 2026-02-26T02:53:43.245 INFO:teuthology.orchestra.run.trial123.stdout: This package will always depend on the latest generic kernel image 2026-02-26T02:53:43.246 INFO:teuthology.orchestra.run.trial123.stdout: available. 2026-02-26T02:53:43.246 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-170-generic Expected: 5.15.0-171-generic 2026-02-26T02:53:43.246 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-26T02:53:43.247 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-26T02:53:43.247 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-26T02:53:43.247 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial123.front.sepia.ceph.com, path=None, version=distro) 2026-02-26T02:53:43.247 DEBUG:teuthology.orchestra.run.trial123:> sudo apt-get clean 2026-02-26T02:53:43.326 DEBUG:teuthology.orchestra.run.trial123:> sudo apt-get update 2026-02-26T02:53:43.759 INFO:teuthology.orchestra.run.trial123.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-26T02:53:43.825 INFO:teuthology.orchestra.run.trial123.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-26T02:53:43.902 INFO:teuthology.orchestra.run.trial123.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-26T02:53:43.976 INFO:teuthology.orchestra.run.trial123.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-26T02:53:44.633 INFO:teuthology.orchestra.run.trial123.stdout:Reading package lists... 2026-02-26T02:53:44.647 DEBUG:teuthology.orchestra.run.trial123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-26T02:53:44.681 INFO:teuthology.orchestra.run.trial123.stdout:Reading package lists... 2026-02-26T02:53:44.795 INFO:teuthology.orchestra.run.trial123.stdout:Building dependency tree... 2026-02-26T02:53:44.795 INFO:teuthology.orchestra.run.trial123.stdout:Reading state information... 2026-02-26T02:53:44.887 INFO:teuthology.orchestra.run.trial123.stdout:linux-image-generic is already the newest version (5.15.0.171.160). 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout:linux-image-generic set to manually installed. 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout:The following packages were automatically installed and are no longer required: 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-26T02:53:44.888 INFO:teuthology.orchestra.run.trial123.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-26T02:53:44.908 INFO:teuthology.orchestra.run.trial123.stdout:0 upgraded, 0 newly installed, 0 to remove and 31 not upgraded. 2026-02-26T02:53:44.910 DEBUG:teuthology.orchestra.run.trial123:> dpkg -s linux-image-generic 2026-02-26T02:53:44.959 INFO:teuthology.orchestra.run.trial123.stdout:Package: linux-image-generic 2026-02-26T02:53:44.959 INFO:teuthology.orchestra.run.trial123.stdout:Status: install ok installed 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Priority: optional 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Section: kernel 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Installed-Size: 22 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Maintainer: Ubuntu Kernel Team 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Architecture: amd64 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Source: linux-meta 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Version: 5.15.0.171.160 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Recommends: thermald 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout:Description: Generic Linux kernel image 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout: This package will always depend on the latest generic kernel image 2026-02-26T02:53:44.960 INFO:teuthology.orchestra.run.trial123.stdout: available. 2026-02-26T02:53:44.960 DEBUG:teuthology.orchestra.run.trial123:> mktemp 2026-02-26T02:53:45.006 INFO:teuthology.orchestra.run.trial123.stdout:/tmp/tmp.fAn5E1gYSO 2026-02-26T02:53:45.006 DEBUG:teuthology.orchestra.run.trial123:> sudo cp /boot/grub/grub.cfg /tmp/tmp.fAn5E1gYSO 2026-02-26T02:53:45.055 DEBUG:teuthology.orchestra.run.trial123:> sudo chmod 0666 /tmp/tmp.fAn5E1gYSO 2026-02-26T02:53:45.153 DEBUG:teuthology.orchestra.remote:trial123:/tmp/tmp.fAn5E1gYSO is 9KB 2026-02-26T02:53:45.161 DEBUG:teuthology.orchestra.run.trial123:> rm -fr /tmp/tmp.fAn5E1gYSO 2026-02-26T02:53:45.166 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-26T02:53:45.215 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:53:45.215 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-26T02:53:45.215 DEBUG:teuthology.orchestra.run.trial123:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-26T02:53:45.272 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-171-generic 2026-02-26T02:53:45.272 DEBUG:teuthology.orchestra.run.trial123:> sudo update-grub 2026-02-26T02:53:45.513 INFO:teuthology.orchestra.run.trial123.stderr:Sourcing file `/etc/default/grub' 2026-02-26T02:53:45.533 INFO:teuthology.orchestra.run.trial123.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-26T02:53:45.533 INFO:teuthology.orchestra.run.trial123.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-26T02:53:45.536 INFO:teuthology.orchestra.run.trial123.stderr:Generating grub configuration file ... 2026-02-26T02:53:45.620 INFO:teuthology.orchestra.run.trial123.stderr:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-26T02:53:45.626 INFO:teuthology.orchestra.run.trial123.stderr:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-26T02:53:45.805 INFO:teuthology.orchestra.run.trial123.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-26T02:53:45.810 INFO:teuthology.orchestra.run.trial123.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-26T02:53:45.853 INFO:teuthology.orchestra.run.trial123.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-26T02:53:45.859 INFO:teuthology.orchestra.run.trial123.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-26T02:53:45.932 INFO:teuthology.orchestra.run.trial123.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-26T02:53:45.933 INFO:teuthology.orchestra.run.trial123.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-26T02:53:45.933 INFO:teuthology.orchestra.run.trial123.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-26T02:53:45.934 INFO:teuthology.orchestra.run.trial123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-26T02:53:45.942 INFO:teuthology.orchestra.run.trial123.stderr:done 2026-02-26T02:53:45.944 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial123 2026-02-26T02:53:45.944 DEBUG:teuthology.orchestra.run.trial123:> sudo bash -c ' 2026-02-26T02:53:45.944 DEBUG:teuthology.orchestra.run.trial123:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-26T02:53:45.944 DEBUG:teuthology.orchestra.run.trial123:> ' 2026-02-26T02:54:15.946 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-26T02:54:15.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial123.front.sepia.ceph.com' 2026-02-26T02:54:15.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T02:55:16.009 DEBUG:teuthology.orchestra.remote:timed out 2026-02-26T02:55:25.019 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial123.front.sepia.ceph.com' 2026-02-26T02:55:25.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T02:55:25.143 DEBUG:teuthology.orchestra.run.trial123:> true 2026-02-26T02:55:25.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial123.front.sepia.ceph.com' 2026-02-26T02:55:25.543 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-171-generic"... 2026-02-26T02:55:25.543 DEBUG:teuthology.orchestra.run.trial123:> uname -r 2026-02-26T02:55:25.586 INFO:teuthology.orchestra.run.trial123.stdout:5.15.0-171-generic 2026-02-26T02:55:25.587 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-171-generic vs 5.15.0-171-generic 2026-02-26T02:55:25.587 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-26T02:55:25.587 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-26T02:55:26.588 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-26T02:55:26.588 DEBUG:teuthology.orchestra.run.trial123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-26T02:55:26.601 INFO:teuthology.orchestra.run.trial123.stdout:ttyS1 2026-02-26T02:55:26.629 DEBUG:teuthology.parallel:result is None 2026-02-26T02:55:26.629 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-26T02:55:26.641 INFO:teuthology.task.internal:Creating test directory... 2026-02-26T02:55:26.642 DEBUG:teuthology.orchestra.run.trial123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-26T02:55:26.674 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-26T02:55:26.685 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-26T02:55:26.699 INFO:teuthology.task.internal:Creating archive directory... 2026-02-26T02:55:26.699 DEBUG:teuthology.orchestra.run.trial123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-26T02:55:26.725 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-26T02:55:26.736 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-26T02:55:26.736 DEBUG:teuthology.orchestra.run.trial123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-26T02:55:26.768 INFO:teuthology.orchestra.run.trial123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:55:26.774 INFO:teuthology.orchestra.run.trial123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:55:26.776 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-26T02:55:26.788 INFO:teuthology.task.internal:Configuring sudo... 2026-02-26T02:55:26.789 DEBUG:teuthology.orchestra.run.trial123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-26T02:55:26.828 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-26T02:55:26.846 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-26T02:55:26.846 DEBUG:teuthology.orchestra.run.trial123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-26T02:55:26.879 DEBUG:teuthology.orchestra.run.trial123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T02:55:26.923 DEBUG:teuthology.orchestra.run.trial123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T02:55:26.971 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:55:26.971 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-26T02:55:27.019 DEBUG:teuthology.orchestra.run.trial123:> sudo service rsyslog restart 2026-02-26T02:55:27.076 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-26T02:55:27.089 INFO:teuthology.task.internal:Starting timer... 2026-02-26T02:55:27.089 INFO:teuthology.run_tasks:Running task pcp... 2026-02-26T02:55:27.110 INFO:teuthology.run_tasks:Running task selinux... 2026-02-26T02:55:27.130 DEBUG:teuthology.task.selinux:Excluding trial123: OS 'ubuntu' does not support SELinux 2026-02-26T02:55:27.130 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-26T02:55:27.130 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-26T02:55:27.130 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-26T02:55:27.130 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-26T02:55:27.146 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-26T02:55:27.158 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-26T02:55:27.159 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-26T02:57:46.880 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial123.front.sepia.ceph.com')] 2026-02-26T02:57:46.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial123.front.sepia.ceph.com' 2026-02-26T02:57:46.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-26T02:57:46.995 DEBUG:teuthology.orchestra.run.trial123:> true 2026-02-26T02:57:47.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial123.front.sepia.ceph.com' 2026-02-26T02:57:47.247 INFO:teuthology.run_tasks:Running task clock... 2026-02-26T02:57:47.263 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-26T02:57:47.264 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-26T02:57:47.264 DEBUG:teuthology.orchestra.run.trial123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T02:57:47.308 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Command line: ntpd -gq 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: ---------------------------------------------------- 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: ntp-4 is maintained by Network Time Foundation, 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: corporation. Support and training for ntp-4 are 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: available at https://www.nwtime.org/support 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: ---------------------------------------------------- 2026-02-26T02:57:47.309 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: proto: precision = 0.020 usec (-25) 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: basedate set to 2022-02-04 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: gps base set to 2022-02-06 (week 2196) 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: restrict ::: KOD does nothing without LIMITED. 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listen and drop on 0 v6wildcard [::]:123 2026-02-26T02:57:47.310 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-26T02:57:47.311 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listen normally on 2 lo 127.0.0.1:123 2026-02-26T02:57:47.311 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listen normally on 3 enp1s0f0 10.20.193.123:123 2026-02-26T02:57:47.311 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listen normally on 4 lo [::1]:123 2026-02-26T02:57:47.311 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:61ca%2]:123 2026-02-26T02:57:47.311 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:47 ntpd[8833]: Listening on routing socket on fd #22 for interface updates 2026-02-26T02:57:48.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:48 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:48.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:48 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:48.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:48 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:48.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:48 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:48.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:48 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:48.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:48 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:49.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:49 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:49.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:49 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:49.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:49 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:49.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:49 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:49.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:49 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:49.313 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:49 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:50.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:50.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:50.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:50.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:50.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:50.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:50.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:50.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:50.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:50.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:50.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:50.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:50 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:51.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:51 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:51.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:51 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:51.313 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:51 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:51.313 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:51 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:51.313 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:51 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:51.313 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:51 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:52.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:52.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:52.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:52.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:52.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:52.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:52.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:52.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:52.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:52.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:52.374 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:52.374 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:52.374 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:52.374 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:52.377 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:52.377 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:52.377 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:52.377 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:52 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:53.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:53 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:53.309 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:53 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:53.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:53 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:53.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:53 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:53.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:53 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:53.312 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:53 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:54.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:54.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:54.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:54.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:54.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:54.310 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:54.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:54.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:54.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:54.329 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:54.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-26T02:57:54.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/rawstats.20260226: Permission denied 2026-02-26T02:57:54.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-26T02:57:54.375 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/peerstats.20260226: Permission denied 2026-02-26T02:57:54.376 INFO:teuthology.orchestra.run.trial123.stdout:26 Feb 02:57:54 ntpd[8833]: ntpd: time set -0.223105 s 2026-02-26T02:57:54.376 INFO:teuthology.orchestra.run.trial123.stdout:ntpd: time set -0.223105s 2026-02-26T02:57:54.376 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-26T02:57:54.376 INFO:teuthology.orchestra.run.trial123.stderr:26 Feb 02:57:54 ntpd[8833]: can't open /var/log/ntpstats/loopstats.20260226: Permission denied 2026-02-26T02:57:54.439 INFO:teuthology.orchestra.run.trial123.stdout: remote refid st t when poll reach delay offset jitter 2026-02-26T02:57:54.439 INFO:teuthology.orchestra.run.trial123.stdout:============================================================================== 2026-02-26T02:57:54.439 INFO:teuthology.orchestra.run.trial123.stdout: 172-233-157-223 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-26T02:57:54.439 INFO:teuthology.orchestra.run.trial123.stdout: 209.177.158.85 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-26T02:57:54.439 INFO:teuthology.orchestra.run.trial123.stdout: srcf-ntp.stanfo .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-26T02:57:54.439 INFO:teuthology.orchestra.run.trial123.stdout: ntp.speculation .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-26T02:57:54.440 INFO:teuthology.run_tasks:Running task install... 2026-02-26T02:57:54.452 DEBUG:teuthology.task.install:project ceph 2026-02-26T02:57:54.452 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5'}} 2026-02-26T02:57:54.452 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5'} 2026-02-26T02:57:54.452 INFO:teuthology.task.install:Using flavor: default 2026-02-26T02:57:54.458 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-26T02:57:54.459 INFO:teuthology.task.install:extra packages: [] 2026-02-26T02:57:54.459 DEBUG:teuthology.orchestra.run.trial123:> sudo apt-key list | grep Ceph 2026-02-26T02:57:54.529 INFO:teuthology.orchestra.run.trial123.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-26T02:57:54.558 INFO:teuthology.orchestra.run.trial123.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-26T02:57:54.558 INFO:teuthology.orchestra.run.trial123.stdout:uid [ unknown] Ceph.com (release key) 2026-02-26T02:57:54.559 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-26T02:57:54.559 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 2026-02-26T02:57:54.708 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default/ 2026-02-26T02:57:54.708 INFO:teuthology.task.install.deb:Package version is 19.2.3-687-g8d724284-1jammy 2026-02-26T02:57:54.828 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:57:54.829 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-26T02:57:54.839 DEBUG:teuthology.orchestra.run.trial123:> sudo apt-get update 2026-02-26T02:57:55.005 INFO:teuthology.orchestra.run.trial123.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy InRelease 2026-02-26T02:57:55.017 INFO:teuthology.orchestra.run.trial123.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy Release [7677 B] 2026-02-26T02:57:55.029 INFO:teuthology.orchestra.run.trial123.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-26T02:57:55.040 INFO:teuthology.orchestra.run.trial123.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-26T02:57:55.280 INFO:teuthology.orchestra.run.trial123.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-26T02:57:55.346 INFO:teuthology.orchestra.run.trial123.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-26T02:57:55.419 INFO:teuthology.orchestra.run.trial123.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-26T02:57:55.491 INFO:teuthology.orchestra.run.trial123.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-26T02:57:55.634 INFO:teuthology.orchestra.run.trial123.stdout:Fetched 25.8 kB in 1s (40.7 kB/s) 2026-02-26T02:57:56.155 INFO:teuthology.orchestra.run.trial123.stdout:Reading package lists... 2026-02-26T02:57:56.168 DEBUG:teuthology.orchestra.run.trial123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-687-g8d724284-1jammy cephadm=19.2.3-687-g8d724284-1jammy ceph-mds=19.2.3-687-g8d724284-1jammy ceph-mgr=19.2.3-687-g8d724284-1jammy ceph-common=19.2.3-687-g8d724284-1jammy ceph-fuse=19.2.3-687-g8d724284-1jammy ceph-test=19.2.3-687-g8d724284-1jammy ceph-volume=19.2.3-687-g8d724284-1jammy radosgw=19.2.3-687-g8d724284-1jammy python3-rados=19.2.3-687-g8d724284-1jammy python3-rgw=19.2.3-687-g8d724284-1jammy python3-cephfs=19.2.3-687-g8d724284-1jammy python3-rbd=19.2.3-687-g8d724284-1jammy libcephfs2=19.2.3-687-g8d724284-1jammy libcephfs-dev=19.2.3-687-g8d724284-1jammy librados2=19.2.3-687-g8d724284-1jammy librbd1=19.2.3-687-g8d724284-1jammy rbd-fuse=19.2.3-687-g8d724284-1jammy 2026-02-26T02:57:56.201 INFO:teuthology.orchestra.run.trial123.stdout:Reading package lists... 2026-02-26T02:57:56.310 INFO:teuthology.orchestra.run.trial123.stdout:Building dependency tree... 2026-02-26T02:57:56.311 INFO:teuthology.orchestra.run.trial123.stdout:Reading state information... 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout:The following packages were automatically installed and are no longer required: 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout:The following additional packages will be installed: 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-26T02:57:56.402 INFO:teuthology.orchestra.run.trial123.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout:Suggested packages: 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout:Recommended packages: 2026-02-26T02:57:56.403 INFO:teuthology.orchestra.run.trial123.stdout: btrfs-tools 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout:The following NEW packages will be installed: 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-26T02:57:56.438 INFO:teuthology.orchestra.run.trial123.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-26T02:57:56.439 INFO:teuthology.orchestra.run.trial123.stdout: xmlstarlet zip 2026-02-26T02:57:56.537 INFO:teuthology.orchestra.run.trial123.stdout:0 upgraded, 108 newly installed, 0 to remove and 31 not upgraded. 2026-02-26T02:57:56.537 INFO:teuthology.orchestra.run.trial123.stdout:Need to get 181 MB of archives. 2026-02-26T02:57:56.537 INFO:teuthology.orchestra.run.trial123.stdout:After this operation, 819 MB of additional disk space will be used. 2026-02-26T02:57:56.537 INFO:teuthology.orchestra.run.trial123.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-687-g8d724284-1jammy [3596 kB] 2026-02-26T02:57:56.549 INFO:teuthology.orchestra.run.trial123.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-26T02:57:56.596 INFO:teuthology.orchestra.run.trial123.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-26T02:57:56.597 INFO:teuthology.orchestra.run.trial123.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-26T02:57:56.607 INFO:teuthology.orchestra.run.trial123.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-26T02:57:56.637 INFO:teuthology.orchestra.run.trial123.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-26T02:57:56.638 INFO:teuthology.orchestra.run.trial123.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-26T02:57:56.640 INFO:teuthology.orchestra.run.trial123.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-687-g8d724284-1jammy [3257 kB] 2026-02-26T02:57:56.644 INFO:teuthology.orchestra.run.trial123.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-26T02:57:56.646 INFO:teuthology.orchestra.run.trial123.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-26T02:57:56.646 INFO:teuthology.orchestra.run.trial123.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-26T02:57:56.647 INFO:teuthology.orchestra.run.trial123.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-26T02:57:56.647 INFO:teuthology.orchestra.run.trial123.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-26T02:57:56.652 INFO:teuthology.orchestra.run.trial123.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-26T02:57:56.653 INFO:teuthology.orchestra.run.trial123.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-26T02:57:56.654 INFO:teuthology.orchestra.run.trial123.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-26T02:57:56.658 INFO:teuthology.orchestra.run.trial123.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-26T02:57:56.658 INFO:teuthology.orchestra.run.trial123.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-26T02:57:56.659 INFO:teuthology.orchestra.run.trial123.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-26T02:57:56.661 INFO:teuthology.orchestra.run.trial123.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-26T02:57:56.667 INFO:teuthology.orchestra.run.trial123.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-26T02:57:56.668 INFO:teuthology.orchestra.run.trial123.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-26T02:57:56.668 INFO:teuthology.orchestra.run.trial123.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-26T02:57:56.669 INFO:teuthology.orchestra.run.trial123.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-26T02:57:56.669 INFO:teuthology.orchestra.run.trial123.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-26T02:57:56.669 INFO:teuthology.orchestra.run.trial123.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-26T02:57:56.677 INFO:teuthology.orchestra.run.trial123.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-26T02:57:56.677 INFO:teuthology.orchestra.run.trial123.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-26T02:57:56.678 INFO:teuthology.orchestra.run.trial123.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-26T02:57:56.678 INFO:teuthology.orchestra.run.trial123.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-26T02:57:56.685 INFO:teuthology.orchestra.run.trial123.stdout:Get:31 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-687-g8d724284-1jammy [979 kB] 2026-02-26T02:57:56.687 INFO:teuthology.orchestra.run.trial123.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-26T02:57:56.687 INFO:teuthology.orchestra.run.trial123.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-26T02:57:56.687 INFO:teuthology.orchestra.run.trial123.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-26T02:57:56.688 INFO:teuthology.orchestra.run.trial123.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-26T02:57:56.688 INFO:teuthology.orchestra.run.trial123.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-26T02:57:56.688 INFO:teuthology.orchestra.run.trial123.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-26T02:57:56.696 INFO:teuthology.orchestra.run.trial123.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-26T02:57:56.697 INFO:teuthology.orchestra.run.trial123.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-26T02:57:56.697 INFO:teuthology.orchestra.run.trial123.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-26T02:57:56.700 INFO:teuthology.orchestra.run.trial123.stdout:Get:41 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-687-g8d724284-1jammy [357 kB] 2026-02-26T02:57:56.700 INFO:teuthology.orchestra.run.trial123.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-26T02:57:56.706 INFO:teuthology.orchestra.run.trial123.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-687-g8d724284-1jammy [32.9 kB] 2026-02-26T02:57:56.706 INFO:teuthology.orchestra.run.trial123.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-26T02:57:56.707 INFO:teuthology.orchestra.run.trial123.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-26T02:57:56.707 INFO:teuthology.orchestra.run.trial123.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-687-g8d724284-1jammy [184 kB] 2026-02-26T02:57:56.707 INFO:teuthology.orchestra.run.trial123.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-26T02:57:56.708 INFO:teuthology.orchestra.run.trial123.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-26T02:57:56.709 INFO:teuthology.orchestra.run.trial123.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-26T02:57:56.710 INFO:teuthology.orchestra.run.trial123.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-687-g8d724284-1jammy [70.1 kB] 2026-02-26T02:57:56.710 INFO:teuthology.orchestra.run.trial123.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-26T02:57:56.710 INFO:teuthology.orchestra.run.trial123.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-687-g8d724284-1jammy [335 kB] 2026-02-26T02:57:56.716 INFO:teuthology.orchestra.run.trial123.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-26T02:57:56.716 INFO:teuthology.orchestra.run.trial123.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-26T02:57:56.716 INFO:teuthology.orchestra.run.trial123.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-687-g8d724284-1jammy [6935 kB] 2026-02-26T02:57:56.717 INFO:teuthology.orchestra.run.trial123.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-26T02:57:56.732 INFO:teuthology.orchestra.run.trial123.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-26T02:57:56.733 INFO:teuthology.orchestra.run.trial123.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-26T02:57:56.733 INFO:teuthology.orchestra.run.trial123.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-26T02:57:56.743 INFO:teuthology.orchestra.run.trial123.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-26T02:57:56.743 INFO:teuthology.orchestra.run.trial123.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-26T02:57:56.743 INFO:teuthology.orchestra.run.trial123.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-26T02:57:56.743 INFO:teuthology.orchestra.run.trial123.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-26T02:57:56.743 INFO:teuthology.orchestra.run.trial123.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-26T02:57:56.744 INFO:teuthology.orchestra.run.trial123.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-26T02:57:56.745 INFO:teuthology.orchestra.run.trial123.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-26T02:57:56.746 INFO:teuthology.orchestra.run.trial123.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-26T02:57:56.754 INFO:teuthology.orchestra.run.trial123.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-26T02:57:56.754 INFO:teuthology.orchestra.run.trial123.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-26T02:57:56.756 INFO:teuthology.orchestra.run.trial123.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-26T02:57:56.757 INFO:teuthology.orchestra.run.trial123.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-26T02:57:56.758 INFO:teuthology.orchestra.run.trial123.stdout:Get:72 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-26T02:57:56.758 INFO:teuthology.orchestra.run.trial123.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-26T02:57:56.764 INFO:teuthology.orchestra.run.trial123.stdout:Get:74 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-26T02:57:56.764 INFO:teuthology.orchestra.run.trial123.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-26T02:57:56.765 INFO:teuthology.orchestra.run.trial123.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-26T02:57:56.765 INFO:teuthology.orchestra.run.trial123.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-26T02:57:56.774 INFO:teuthology.orchestra.run.trial123.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-26T02:57:56.775 INFO:teuthology.orchestra.run.trial123.stdout:Get:79 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-26T02:57:56.776 INFO:teuthology.orchestra.run.trial123.stdout:Get:80 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-26T02:57:56.777 INFO:teuthology.orchestra.run.trial123.stdout:Get:81 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-26T02:57:56.780 INFO:teuthology.orchestra.run.trial123.stdout:Get:82 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-26T02:57:56.781 INFO:teuthology.orchestra.run.trial123.stdout:Get:83 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-26T02:57:56.784 INFO:teuthology.orchestra.run.trial123.stdout:Get:84 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-26T02:57:56.785 INFO:teuthology.orchestra.run.trial123.stdout:Get:85 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-26T02:57:56.785 INFO:teuthology.orchestra.run.trial123.stdout:Get:86 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-26T02:57:56.831 INFO:teuthology.orchestra.run.trial123.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-687-g8d724284-1jammy [112 kB] 2026-02-26T02:57:56.832 INFO:teuthology.orchestra.run.trial123.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-687-g8d724284-1jammy [470 kB] 2026-02-26T02:57:56.838 INFO:teuthology.orchestra.run.trial123.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-687-g8d724284-1jammy [26.5 MB] 2026-02-26T02:57:57.278 INFO:teuthology.orchestra.run.trial123.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-687-g8d724284-1jammy [5178 kB] 2026-02-26T02:57:57.359 INFO:teuthology.orchestra.run.trial123.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-687-g8d724284-1jammy [248 kB] 2026-02-26T02:57:57.362 INFO:teuthology.orchestra.run.trial123.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-687-g8d724284-1jammy [125 kB] 2026-02-26T02:57:57.364 INFO:teuthology.orchestra.run.trial123.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-687-g8d724284-1jammy [1081 kB] 2026-02-26T02:57:57.382 INFO:teuthology.orchestra.run.trial123.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-687-g8d724284-1jammy [6239 kB] 2026-02-26T02:57:57.485 INFO:teuthology.orchestra.run.trial123.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-687-g8d724284-1jammy [23.0 MB] 2026-02-26T02:57:57.864 INFO:teuthology.orchestra.run.trial123.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-687-g8d724284-1jammy [14.2 kB] 2026-02-26T02:57:57.864 INFO:teuthology.orchestra.run.trial123.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-687-g8d724284-1jammy [1173 kB] 2026-02-26T02:57:57.880 INFO:teuthology.orchestra.run.trial123.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-687-g8d724284-1jammy [2503 kB] 2026-02-26T02:57:57.921 INFO:teuthology.orchestra.run.trial123.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-687-g8d724284-1jammy [798 kB] 2026-02-26T02:57:57.934 INFO:teuthology.orchestra.run.trial123.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-687-g8d724284-1jammy [157 kB] 2026-02-26T02:57:57.937 INFO:teuthology.orchestra.run.trial123.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-687-g8d724284-1jammy [2396 kB] 2026-02-26T02:57:57.976 INFO:teuthology.orchestra.run.trial123.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-687-g8d724284-1jammy [8625 kB] 2026-02-26T02:57:58.119 INFO:teuthology.orchestra.run.trial123.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-687-g8d724284-1jammy [14.3 kB] 2026-02-26T02:57:58.119 INFO:teuthology.orchestra.run.trial123.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-687-g8d724284-1jammy [55.6 MB] 2026-02-26T02:57:59.037 INFO:teuthology.orchestra.run.trial123.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-687-g8d724284-1jammy [135 kB] 2026-02-26T02:57:59.038 INFO:teuthology.orchestra.run.trial123.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-687-g8d724284-1jammy [41.1 kB] 2026-02-26T02:57:59.038 INFO:teuthology.orchestra.run.trial123.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-687-g8d724284-1jammy [13.8 MB] 2026-02-26T02:57:59.258 INFO:teuthology.orchestra.run.trial123.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-yuri9-testing-2026-02-25-1600-squid/8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-687-g8d724284-1jammy [92.1 kB] 2026-02-26T02:57:59.595 INFO:teuthology.orchestra.run.trial123.stdout:Fetched 181 MB in 3s (64.3 MB/s) 2026-02-26T02:57:59.611 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-26T02:57:59.776 INFO:teuthology.orchestra.run.trial123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 202926 files and directories currently installed.) 2026-02-26T02:57:59.779 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-26T02:57:59.780 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-26T02:57:59.800 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-26T02:57:59.815 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-26T02:57:59.815 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-26T02:57:59.832 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-26T02:57:59.846 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-26T02:57:59.847 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-26T02:57:59.865 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-26T02:57:59.874 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-26T02:57:59.879 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-26T02:57:59.912 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-26T02:57:59.922 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-26T02:57:59.922 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-26T02:57:59.939 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-26T02:57:59.948 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-26T02:57:59.949 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-26T02:57:59.969 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-26T02:57:59.979 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-26T02:57:59.979 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-26T02:57:59.996 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package librados2. 2026-02-26T02:58:00.006 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../007-librados2_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.006 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking librados2 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.046 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libnbd0. 2026-02-26T02:58:00.056 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-26T02:58:00.056 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-26T02:58:00.072 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package librbd1. 2026-02-26T02:58:00.082 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../009-librbd1_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.082 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking librbd1 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.123 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libcephfs2. 2026-02-26T02:58:00.133 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.133 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libcephfs2 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.155 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-rados. 2026-02-26T02:58:00.165 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../011-python3-rados_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.165 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-rados (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.182 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-26T02:58:00.192 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:00.192 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-ceph-argparse (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.207 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-cephfs. 2026-02-26T02:58:00.217 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.217 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-cephfs (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.233 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-26T02:58:00.243 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:00.248 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-ceph-common (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.261 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-26T02:58:00.271 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-26T02:58:00.272 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-26T02:58:00.288 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-prettytable. 2026-02-26T02:58:00.298 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-26T02:58:00.299 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-26T02:58:00.313 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-rbd. 2026-02-26T02:58:00.323 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.324 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-rbd (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.341 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-26T02:58:00.351 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-26T02:58:00.351 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-26T02:58:00.370 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-26T02:58:00.380 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-26T02:58:00.381 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-26T02:58:00.401 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-26T02:58:00.410 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-26T02:58:00.411 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-26T02:58:00.429 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package lua5.1. 2026-02-26T02:58:00.439 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-26T02:58:00.440 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-26T02:58:00.461 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package lua-any. 2026-02-26T02:58:00.470 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-26T02:58:00.471 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-26T02:58:00.486 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package zip. 2026-02-26T02:58:00.496 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-26T02:58:00.496 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking zip (3.0-12build2) ... 2026-02-26T02:58:00.512 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package unzip. 2026-02-26T02:58:00.522 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-26T02:58:00.523 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-26T02:58:00.540 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package luarocks. 2026-02-26T02:58:00.550 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-26T02:58:00.550 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-26T02:58:00.589 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package librgw2. 2026-02-26T02:58:00.598 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../026-librgw2_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.599 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking librgw2 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.674 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-rgw. 2026-02-26T02:58:00.684 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.685 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-rgw (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.701 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-26T02:58:00.710 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-26T02:58:00.711 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-26T02:58:00.726 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libradosstriper1. 2026-02-26T02:58:00.736 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.737 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libradosstriper1 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.756 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-common. 2026-02-26T02:58:00.766 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../030-ceph-common_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.766 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-common (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:00.973 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-base. 2026-02-26T02:58:00.984 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../031-ceph-base_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:00.989 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-base (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:01.053 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-26T02:58:01.062 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-26T02:58:01.063 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-26T02:58:01.078 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-cheroot. 2026-02-26T02:58:01.089 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-26T02:58:01.089 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-26T02:58:01.106 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-26T02:58:01.116 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-26T02:58:01.116 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-26T02:58:01.132 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-26T02:58:01.142 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-26T02:58:01.142 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-26T02:58:01.158 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-26T02:58:01.169 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-26T02:58:01.169 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-26T02:58:01.184 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-tempora. 2026-02-26T02:58:01.194 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-26T02:58:01.195 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-26T02:58:01.210 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-portend. 2026-02-26T02:58:01.220 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-26T02:58:01.221 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-26T02:58:01.236 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-26T02:58:01.246 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-26T02:58:01.247 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-26T02:58:01.261 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-26T02:58:01.271 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-26T02:58:01.272 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-26T02:58:01.296 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-natsort. 2026-02-26T02:58:01.306 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-26T02:58:01.306 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-26T02:58:01.324 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-logutils. 2026-02-26T02:58:01.333 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-26T02:58:01.334 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-26T02:58:01.349 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-mako. 2026-02-26T02:58:01.359 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-26T02:58:01.360 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-26T02:58:01.377 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-26T02:58:01.387 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-26T02:58:01.387 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-26T02:58:01.402 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-26T02:58:01.412 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-26T02:58:01.413 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-26T02:58:01.428 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-webob. 2026-02-26T02:58:01.438 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-26T02:58:01.438 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-26T02:58:01.456 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-waitress. 2026-02-26T02:58:01.465 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-26T02:58:01.467 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-26T02:58:01.484 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-tempita. 2026-02-26T02:58:01.493 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-26T02:58:01.494 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-26T02:58:01.509 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-paste. 2026-02-26T02:58:01.519 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-26T02:58:01.519 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-26T02:58:01.548 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-26T02:58:01.557 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-26T02:58:01.558 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-26T02:58:01.572 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-26T02:58:01.582 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-26T02:58:01.582 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-26T02:58:01.598 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-webtest. 2026-02-26T02:58:01.608 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-26T02:58:01.608 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-26T02:58:01.625 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pecan. 2026-02-26T02:58:01.634 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-26T02:58:01.635 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-26T02:58:01.660 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-26T02:58:01.670 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-26T02:58:01.670 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-26T02:58:01.691 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-26T02:58:01.700 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:01.701 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mgr-modules-core (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:01.730 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-26T02:58:01.740 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:01.741 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:01.757 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mgr. 2026-02-26T02:58:01.766 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:01.766 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mgr (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:01.790 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mon. 2026-02-26T02:58:01.800 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:01.800 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mon (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:01.858 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-26T02:58:01.867 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-26T02:58:01.868 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-26T02:58:01.884 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-osd. 2026-02-26T02:58:01.893 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:01.894 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-osd (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.052 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph. 2026-02-26T02:58:02.062 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../061-ceph_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:02.062 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.077 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-fuse. 2026-02-26T02:58:02.087 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:02.087 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-fuse (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.111 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mds. 2026-02-26T02:58:02.122 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:02.122 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mds (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.156 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package cephadm. 2026-02-26T02:58:02.165 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../064-cephadm_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:02.166 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking cephadm (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.182 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-26T02:58:02.192 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-26T02:58:02.192 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-26T02:58:02.214 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-26T02:58:02.224 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:02.225 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mgr-cephadm (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.244 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-26T02:58:02.254 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-26T02:58:02.255 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-26T02:58:02.270 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-routes. 2026-02-26T02:58:02.280 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-26T02:58:02.281 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-26T02:58:02.301 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-26T02:58:02.312 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:02.312 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mgr-dashboard (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.514 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-26T02:58:02.525 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-26T02:58:02.525 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-26T02:58:02.564 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-joblib. 2026-02-26T02:58:02.574 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-26T02:58:02.575 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-26T02:58:02.602 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-26T02:58:02.612 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-26T02:58:02.613 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-26T02:58:02.628 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-sklearn. 2026-02-26T02:58:02.638 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-26T02:58:02.639 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-26T02:58:02.721 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-26T02:58:02.732 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:02.732 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:02.900 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-cachetools. 2026-02-26T02:58:02.911 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-26T02:58:02.911 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-26T02:58:02.927 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-rsa. 2026-02-26T02:58:02.937 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-26T02:58:02.937 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-26T02:58:02.955 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-google-auth. 2026-02-26T02:58:02.966 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-26T02:58:02.967 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-26T02:58:02.984 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-26T02:58:02.994 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-26T02:58:02.995 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-26T02:58:03.012 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-websocket. 2026-02-26T02:58:03.022 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-26T02:58:03.023 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-26T02:58:03.041 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-26T02:58:03.051 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-26T02:58:03.051 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-26T02:58:03.145 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-26T02:58:03.157 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:03.157 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:03.173 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-26T02:58:03.182 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-26T02:58:03.189 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-26T02:58:03.206 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-26T02:58:03.217 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-26T02:58:03.217 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-26T02:58:03.233 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package jq. 2026-02-26T02:58:03.243 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-26T02:58:03.244 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-26T02:58:03.259 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package socat. 2026-02-26T02:58:03.270 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-26T02:58:03.270 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-26T02:58:03.289 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package xmlstarlet. 2026-02-26T02:58:03.299 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-26T02:58:03.300 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-26T02:58:03.335 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-test. 2026-02-26T02:58:03.346 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../087-ceph-test_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:03.346 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-test (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:03.842 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package ceph-volume. 2026-02-26T02:58:03.853 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-687-g8d724284-1jammy_all.deb ... 2026-02-26T02:58:03.853 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking ceph-volume (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:03.877 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-26T02:58:03.888 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:03.888 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking libcephfs-dev (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:03.904 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-26T02:58:03.914 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-26T02:58:03.915 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-26T02:58:03.936 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-26T02:58:03.947 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-26T02:58:03.947 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-26T02:58:03.966 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package nvme-cli. 2026-02-26T02:58:03.976 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-26T02:58:03.977 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-26T02:58:04.008 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package pkg-config. 2026-02-26T02:58:04.019 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-26T02:58:04.019 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-26T02:58:04.036 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-26T02:58:04.047 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-26T02:58:04.047 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-26T02:58:04.077 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-26T02:58:04.087 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-26T02:58:04.088 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-26T02:58:04.104 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pastescript. 2026-02-26T02:58:04.115 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-26T02:58:04.115 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-26T02:58:04.133 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pluggy. 2026-02-26T02:58:04.143 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-26T02:58:04.144 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-26T02:58:04.160 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-psutil. 2026-02-26T02:58:04.171 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-26T02:58:04.171 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-26T02:58:04.190 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-py. 2026-02-26T02:58:04.200 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-26T02:58:04.201 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-26T02:58:04.222 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pygments. 2026-02-26T02:58:04.233 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-26T02:58:04.233 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-26T02:58:04.275 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-26T02:58:04.285 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-26T02:58:04.286 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-26T02:58:04.302 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-toml. 2026-02-26T02:58:04.312 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-26T02:58:04.313 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-26T02:58:04.328 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-pytest. 2026-02-26T02:58:04.339 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-26T02:58:04.339 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-26T02:58:04.361 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package python3-simplejson. 2026-02-26T02:58:04.371 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-26T02:58:04.372 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-26T02:58:04.390 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-26T02:58:04.400 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-26T02:58:04.401 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-26T02:58:04.470 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package radosgw. 2026-02-26T02:58:04.480 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../106-radosgw_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:04.481 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking radosgw (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:04.596 INFO:teuthology.orchestra.run.trial123.stdout:Selecting previously unselected package rbd-fuse. 2026-02-26T02:58:04.606 INFO:teuthology.orchestra.run.trial123.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-687-g8d724284-1jammy_amd64.deb ... 2026-02-26T02:58:04.606 INFO:teuthology.orchestra.run.trial123.stdout:Unpacking rbd-fuse (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:04.636 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-26T02:58:04.702 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-26T02:58:04.703 INFO:teuthology.orchestra.run.trial123.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-26T02:58:04.774 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-26T02:58:05.050 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-26T02:58:05.447 INFO:teuthology.orchestra.run.trial123.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-26T02:58:05.458 INFO:teuthology.orchestra.run.trial123.stdout:Setting up cephadm (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:05.502 INFO:teuthology.orchestra.run.trial123.stdout:Adding system user cephadm....done 2026-02-26T02:58:05.511 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-26T02:58:05.582 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-26T02:58:05.655 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-26T02:58:05.655 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-26T02:58:05.727 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-26T02:58:05.803 INFO:teuthology.orchestra.run.trial123.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-26T02:58:05.804 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-26T02:58:05.900 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-26T02:58:06.022 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-26T02:58:06.098 INFO:teuthology.orchestra.run.trial123.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-26T02:58:06.108 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-26T02:58:06.184 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-26T02:58:06.257 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-ceph-argparse (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:06.326 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-26T02:58:06.326 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-26T02:58:06.328 INFO:teuthology.orchestra.run.trial123.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-26T02:58:06.328 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-26T02:58:06.329 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-26T02:58:06.330 INFO:teuthology.orchestra.run.trial123.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-26T02:58:06.347 INFO:teuthology.orchestra.run.trial123.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-26T02:58:06.349 INFO:teuthology.orchestra.run.trial123.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-26T02:58:06.350 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-26T02:58:06.351 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-26T02:58:06.473 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-26T02:58:06.553 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-26T02:58:06.634 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-26T02:58:06.723 INFO:teuthology.orchestra.run.trial123.stdout:Setting up zip (3.0-12build2) ... 2026-02-26T02:58:06.724 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-26T02:58:06.968 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-26T02:58:07.044 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-26T02:58:07.045 INFO:teuthology.orchestra.run.trial123.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-26T02:58:07.046 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-26T02:58:07.138 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-26T02:58:07.261 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-26T02:58:07.387 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-26T02:58:07.468 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-26T02:58:07.572 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-26T02:58:07.644 INFO:teuthology.orchestra.run.trial123.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-26T02:58:07.645 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-ceph-common (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:07.726 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-26T02:58:08.218 INFO:teuthology.orchestra.run.trial123.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-26T02:58:08.240 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-26T02:58:08.244 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-26T02:58:08.323 INFO:teuthology.orchestra.run.trial123.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-26T02:58:08.324 INFO:teuthology.orchestra.run.trial123.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-26T02:58:08.325 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-26T02:58:08.401 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-26T02:58:08.478 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-26T02:58:08.479 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-26T02:58:08.560 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-26T02:58:08.632 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-26T02:58:08.708 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-26T02:58:08.784 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-26T02:58:08.856 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-26T02:58:08.933 INFO:teuthology.orchestra.run.trial123.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-26T02:58:08.934 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-26T02:58:09.017 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-26T02:58:09.018 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-26T02:58:09.096 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-26T02:58:09.174 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-26T02:58:09.259 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-26T02:58:09.332 INFO:teuthology.orchestra.run.trial123.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-26T02:58:09.333 INFO:teuthology.orchestra.run.trial123.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-26T02:58:09.334 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-26T02:58:09.335 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-26T02:58:09.472 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-26T02:58:09.547 INFO:teuthology.orchestra.run.trial123.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-26T02:58:09.548 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-26T02:58:09.622 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-26T02:58:09.624 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-26T02:58:09.705 INFO:teuthology.orchestra.run.trial123.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-26T02:58:09.706 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-26T02:58:09.790 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-26T02:58:09.922 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-26T02:58:10.011 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-26T02:58:10.124 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-26T02:58:10.125 INFO:teuthology.orchestra.run.trial123.stdout:Setting up librados2 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.126 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libsqlite3-mod-ceph (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.127 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-26T02:58:10.644 INFO:teuthology.orchestra.run.trial123.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-26T02:58:10.655 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libcephfs2 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.656 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libradosstriper1 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.656 INFO:teuthology.orchestra.run.trial123.stdout:Setting up librbd1 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.657 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mgr-modules-core (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.658 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-fuse (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:10.730 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-26T02:58:10.730 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-26T02:58:11.128 INFO:teuthology.orchestra.run.trial123.stdout:Setting up libcephfs-dev (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.129 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-rados (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.130 INFO:teuthology.orchestra.run.trial123.stdout:Setting up librgw2 (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.130 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-rbd (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.131 INFO:teuthology.orchestra.run.trial123.stdout:Setting up rbd-fuse (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.132 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-rgw (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.133 INFO:teuthology.orchestra.run.trial123.stdout:Setting up python3-cephfs (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.134 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-common (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.162 INFO:teuthology.orchestra.run.trial123.stdout:Adding group ceph....done 2026-02-26T02:58:11.198 INFO:teuthology.orchestra.run.trial123.stdout:Adding system user ceph....done 2026-02-26T02:58:11.209 INFO:teuthology.orchestra.run.trial123.stdout:Setting system user ceph properties....done 2026-02-26T02:58:11.215 INFO:teuthology.orchestra.run.trial123.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-26T02:58:11.302 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-26T02:58:11.579 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-26T02:58:11.942 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-test (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:11.943 INFO:teuthology.orchestra.run.trial123.stdout:Setting up radosgw (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:12.205 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-26T02:58:12.206 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-26T02:58:12.577 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-base (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:12.655 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-26T02:58:13.010 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mds (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:13.069 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-26T02:58:13.069 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-26T02:58:13.426 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mgr (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:13.503 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-26T02:58:13.504 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-26T02:58:13.862 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-osd (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:13.948 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-26T02:58:13.949 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-26T02:58:14.324 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mgr-k8sevents (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.325 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.335 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mon (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.407 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-26T02:58:14.407 INFO:teuthology.orchestra.run.trial123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-26T02:58:14.780 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mgr-cephadm (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.789 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.790 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-mgr-dashboard (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.802 INFO:teuthology.orchestra.run.trial123.stdout:Setting up ceph-volume (19.2.3-687-g8d724284-1jammy) ... 2026-02-26T02:58:14.913 INFO:teuthology.orchestra.run.trial123.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-26T02:58:14.919 INFO:teuthology.orchestra.run.trial123.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-26T02:58:14.938 INFO:teuthology.orchestra.run.trial123.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-26T02:58:15.025 INFO:teuthology.orchestra.run.trial123.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout:Running kernel seems to be up-to-date. 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout:The processor microcode seems to be up-to-date. 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout:No services need to be restarted. 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout:No containers need to be restarted. 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout:No user sessions are running outdated binaries. 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout: 2026-02-26T02:58:15.514 INFO:teuthology.orchestra.run.trial123.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-26T02:58:16.343 INFO:teuthology.orchestra.run.trial123.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-26T02:58:16.349 DEBUG:teuthology.parallel:result is None 2026-02-26T02:58:16.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 2026-02-26T02:58:16.495 DEBUG:teuthology.orchestra.run.trial123:> dpkg-query -W -f '${Version}' ceph 2026-02-26T02:58:16.505 INFO:teuthology.orchestra.run.trial123.stdout:19.2.3-687-g8d724284-1jammy 2026-02-26T02:58:16.506 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-687-g8d724284-1jammy 2026-02-26T02:58:16.506 INFO:teuthology.task.install:The correct ceph version 19.2.3-687-g8d724284-1jammy is installed. 2026-02-26T02:58:16.508 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-26T02:58:16.508 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:58:16.509 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-26T02:58:16.554 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-26T02:58:16.555 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:58:16.555 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/usr/bin/daemon-helper 2026-02-26T02:58:16.605 DEBUG:teuthology.orchestra.run.trial123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-26T02:58:16.653 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-26T02:58:16.653 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:58:16.653 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-26T02:58:16.702 DEBUG:teuthology.orchestra.run.trial123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-26T02:58:16.753 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-26T02:58:16.753 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:58:16.753 DEBUG:teuthology.orchestra.run.trial123:> sudo dd of=/usr/bin/stdin-killer 2026-02-26T02:58:16.803 DEBUG:teuthology.orchestra.run.trial123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-26T02:58:16.854 INFO:teuthology.run_tasks:Running task workunit... 2026-02-26T02:58:16.873 INFO:tasks.workunit:Pulling workunits from ref 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 2026-02-26T02:58:16.873 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-26T02:58:16.873 INFO:tasks.workunit:timeout=3h 2026-02-26T02:58:16.873 INFO:tasks.workunit:cleanup=True 2026-02-26T02:58:16.873 DEBUG:teuthology.orchestra.run.trial123:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-26T02:58:16.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T02:58:16.898 INFO:teuthology.orchestra.run.trial123.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-26T02:58:16.899 DEBUG:teuthology.orchestra.run.trial123:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-26T02:58:16.946 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-26T02:58:16.946 DEBUG:teuthology.orchestra.run.trial123:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-26T02:58:16.990 DEBUG:teuthology.orchestra.run.trial123:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 2026-02-26T02:58:17.040 INFO:tasks.workunit.client.0.trial123.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-26T02:58:38.687 INFO:tasks.workunit.client.0.trial123.stderr:Note: switching to '8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5'. 2026-02-26T02:58:38.687 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.687 INFO:tasks.workunit.client.0.trial123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-26T02:58:38.688 INFO:tasks.workunit.client.0.trial123.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-26T02:58:38.688 INFO:tasks.workunit.client.0.trial123.stderr:state without impacting any branches by switching back to a branch. 2026-02-26T02:58:38.688 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.688 INFO:tasks.workunit.client.0.trial123.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-26T02:58:38.688 INFO:tasks.workunit.client.0.trial123.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-26T02:58:38.689 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.689 INFO:tasks.workunit.client.0.trial123.stderr: git switch -c 2026-02-26T02:58:38.689 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.689 INFO:tasks.workunit.client.0.trial123.stderr:Or undo this operation with: 2026-02-26T02:58:38.689 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.689 INFO:tasks.workunit.client.0.trial123.stderr: git switch - 2026-02-26T02:58:38.690 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.690 INFO:tasks.workunit.client.0.trial123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-26T02:58:38.690 INFO:tasks.workunit.client.0.trial123.stderr: 2026-02-26T02:58:38.690 INFO:tasks.workunit.client.0.trial123.stderr:HEAD is now at 8d724284d6d Merge branch 'wip-ifed-fix-vselector-math-squi' of https://github.com/ifed01/ceph into wip-yuri9-testing-2026-02-25-1600-squid 2026-02-26T02:58:38.698 DEBUG:teuthology.orchestra.run.trial123:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-26T02:58:38.744 DEBUG:teuthology.orchestra.run.trial123:> set -ex 2026-02-26T02:58:38.744 DEBUG:teuthology.orchestra.run.trial123:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-26T02:58:38.791 INFO:tasks.workunit:Running workunits matching osd on client.0... 2026-02-26T02:58:38.792 INFO:tasks.workunit:Running workunit osd/bad-inc-map.sh... 2026-02-26T02:58:38.792 DEBUG:teuthology.orchestra.run.trial123:workunit test osd/bad-inc-map.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh 2026-02-26T02:58:38.844 INFO:tasks.workunit.client.0.trial123.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/bad-inc-map 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:8: run: local dir=td/bad-inc-map 2026-02-26T02:58:38.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:9: run: shift 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: export CEPH_MON=127.0.0.1:50755 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: CEPH_MON=127.0.0.1:50755 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:12: run: export CEPH_ARGS 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: uuidgen 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: CEPH_ARGS+='--fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none ' 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:14: run: CEPH_ARGS+='--mon-host=127.0.0.1:50755 ' 2026-02-26T02:58:38.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:15: run: set -e 2026-02-26T02:58:38.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: set 2026-02-26T02:58:38.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: local funcs=TEST_bad_inc_map 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:18: run: for func in $funcs 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:19: run: setup td/bad-inc-map 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/bad-inc-map 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/bad-inc-map 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T02:58:38.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T02:58:38.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T02:58:38.866 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T02:58:38.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T02:58:38.867 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T02:58:38.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T02:58:38.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T02:58:38.868 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T02:58:38.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:58:38.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T02:58:38.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T02:58:38.869 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:58:38.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T02:58:38.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T02:58:38.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-26T02:58:38.873 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T02:58:38.873 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:38.873 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:38.873 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11865 2026-02-26T02:58:38.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T02:58:38.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T02:58:38.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/bad-inc-map 2026-02-26T02:58:38.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T02:58:38.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:38.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:38.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.11865 2026-02-26T02:58:38.878 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/bad-inc-map 1' TERM HUP INT 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:20: run: TEST_bad_inc_map td/bad-inc-map 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:26: TEST_bad_inc_map: local dir=td/bad-inc-map 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:28: TEST_bad_inc_map: run_mon td/bad-inc-map a 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/bad-inc-map 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/bad-inc-map/a 2026-02-26T02:58:38.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/bad-inc-map/a --run-dir=td/bad-inc-map 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:38.910 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:38.911 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/bad-inc-map/a '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --mon-cluster-log-file=td/bad-inc-map/log --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T02:58:38.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T02:58:38.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T02:58:38.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T02:58:38.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T02:58:38.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T02:58:38.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:38.940 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.11865/ceph-mon.a.asok 2026-02-26T02:58:38.941 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T02:58:38.941 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.11865/ceph-mon.a.asok config get fsid 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T02:58:38.997 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T02:58:38.998 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T02:58:38.998 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:38.998 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:38.998 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.11865/ceph-mon.a.asok 2026-02-26T02:58:38.998 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T02:58:38.998 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.11865/ceph-mon.a.asok config get mon_host 2026-02-26T02:58:39.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:29: TEST_bad_inc_map: run_mgr td/bad-inc-map x 2026-02-26T02:58:39.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/bad-inc-map 2026-02-26T02:58:39.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T02:58:39.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T02:58:39.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T02:58:39.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/bad-inc-map/x 2026-02-26T02:58:39.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T02:58:39.161 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T02:58:39.161 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:58:39.161 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:58:39.161 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:58:39.162 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:39.162 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:39.162 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:39.162 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T02:58:39.163 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/bad-inc-map/x '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:30: TEST_bad_inc_map: run_osd td/bad-inc-map 0 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/0 2026-02-26T02:58:39.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 ' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/0' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/0/journal' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:58:39.187 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T02:58:39.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/0 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T02:58:39.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6c3c37f7-7247-4a64-9ece-6bd413359701 2026-02-26T02:58:39.190 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 6c3c37f7-7247-4a64-9ece-6bd413359701 2026-02-26T02:58:39.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 6c3c37f7-7247-4a64-9ece-6bd413359701' 2026-02-26T02:58:39.190 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T02:58:39.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDftp9p+0XeCxAAWHuWiPBJ+NTKDlrZu1lYRw== 2026-02-26T02:58:39.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDftp9p+0XeCxAAWHuWiPBJ+NTKDlrZu1lYRw=="}' 2026-02-26T02:58:39.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6c3c37f7-7247-4a64-9ece-6bd413359701 -i td/bad-inc-map/0/new.json 2026-02-26T02:58:39.314 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:58:39.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/0/new.json 2026-02-26T02:58:39.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDftp9p+0XeCxAAWHuWiPBJ+NTKDlrZu1lYRw== --osd-uuid 6c3c37f7-7247-4a64-9ece-6bd413359701 2026-02-26T02:58:39.340 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:39.335+0000 7fecb5a928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:39.341 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:39.335+0000 7fecb5a928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:39.343 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:39.339+0000 7fecb5a928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:39.343 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:39.339+0000 7fecb5a928c0 -1 bdev(0x55645c2f1c00 td/bad-inc-map/0/block) open stat got: (1) Operation not permitted 2026-02-26T02:58:39.343 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:39.339+0000 7fecb5a928c0 -1 bluestore(td/bad-inc-map/0) _read_fsid unparsable uuid 2026-02-26T02:58:42.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/0/keyring 2026-02-26T02:58:42.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T02:58:42.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T02:58:42.209 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T02:58:42.209 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T02:58:42.478 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T02:58:42.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T02:58:42.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T02:58:42.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T02:58:42.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T02:58:42.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T02:58:42.499 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:42.494+0000 7f2294ef48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:42.500 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:42.494+0000 7f2294ef48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:42.502 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:42.494+0000 7f2294ef48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:42.677 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:58:42.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:58:42.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:42.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:58:42.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:43.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:43.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:43.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T02:58:43.880 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:58:43.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:43.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:58:43.995 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:43.990+0000 7f2294ef48c0 -1 Falling back to public interface 2026-02-26T02:58:44.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:45.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:45.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:45.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T02:58:45.080 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:58:45.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:45.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:58:45.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:46.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:46.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:46.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T02:58:46.284 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T02:58:46.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:46.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:58:46.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:47.199 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:47.194+0000 7f2294ef48c0 -1 osd.0 0 log_to_monitors true 2026-02-26T02:58:47.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:47.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:47.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T02:58:47.476 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T02:58:47.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:47.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:58:47.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:48.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:48.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:48.685 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T02:58:48.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T02:58:48.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:48.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:58:48.882 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4073897922,v1:127.0.0.1:6803/4073897922] [v2:127.0.0.1:6804/4073897922,v1:127.0.0.1:6805/4073897922] exists,up 6c3c37f7-7247-4a64-9ece-6bd413359701 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:31: TEST_bad_inc_map: run_osd td/bad-inc-map 1 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T02:58:48.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/1 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 ' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/1' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/1/journal' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T02:58:48.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T02:58:48.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/1 2026-02-26T02:58:48.888 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T02:58:48.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f0f662ce-921a-40fd-a288-96b40b8f449d 2026-02-26T02:58:48.890 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 f0f662ce-921a-40fd-a288-96b40b8f449d 2026-02-26T02:58:48.891 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 f0f662ce-921a-40fd-a288-96b40b8f449d' 2026-02-26T02:58:48.891 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T02:58:48.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDotp9pdFI/NRAAJ4m9/7M0LZJ6hC+m9h/f5g== 2026-02-26T02:58:48.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDotp9pdFI/NRAAJ4m9/7M0LZJ6hC+m9h/f5g=="}' 2026-02-26T02:58:48.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f0f662ce-921a-40fd-a288-96b40b8f449d -i td/bad-inc-map/1/new.json 2026-02-26T02:58:49.099 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:58:49.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/1/new.json 2026-02-26T02:58:49.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDotp9pdFI/NRAAJ4m9/7M0LZJ6hC+m9h/f5g== --osd-uuid f0f662ce-921a-40fd-a288-96b40b8f449d 2026-02-26T02:58:49.125 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:49.118+0000 7fb8765768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:49.126 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:49.122+0000 7fb8765768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:49.127 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:49.122+0000 7fb8765768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:49.128 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:49.122+0000 7fb8765768c0 -1 bdev(0x5559183ffc00 td/bad-inc-map/1/block) open stat got: (1) Operation not permitted 2026-02-26T02:58:49.128 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:49.122+0000 7fb8765768c0 -1 bluestore(td/bad-inc-map/1) _read_fsid unparsable uuid 2026-02-26T02:58:51.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/1/keyring 2026-02-26T02:58:51.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T02:58:51.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T02:58:51.411 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T02:58:51.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T02:58:51.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T02:58:51.683 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T02:58:51.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T02:58:51.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T02:58:51.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T02:58:51.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T02:58:51.703 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:51.698+0000 7f92baeba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:51.704 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:51.698+0000 7f92baeba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:51.705 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:51.698+0000 7f92baeba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:51.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T02:58:51.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:58:51.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T02:58:51.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:58:51.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:58:51.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:51.881 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:58:51.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:58:51.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:51.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:58:52.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:53.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:53.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:53.087 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:58:53.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T02:58:53.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:53.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:58:53.203 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:53.198+0000 7f92baeba8c0 -1 Falling back to public interface 2026-02-26T02:58:53.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:54.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:54.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:54.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T02:58:54.287 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:58:54.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:54.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:58:54.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:55.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:55.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:55.493 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T02:58:55.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T02:58:55.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:55.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:58:55.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:56.421 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:56.414+0000 7f92baeba8c0 -1 osd.1 0 log_to_monitors true 2026-02-26T02:58:56.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:56.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:56.693 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T02:58:56.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T02:58:56.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:56.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:58:56.902 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:58:57.903 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:58:57.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:58:57.904 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T02:58:57.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T02:58:57.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:58:57.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:58:58.100 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4276654763,v1:127.0.0.1:6811/4276654763] [v2:127.0.0.1:6812/4276654763,v1:127.0.0.1:6813/4276654763] exists,up f0f662ce-921a-40fd-a288-96b40b8f449d 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:32: TEST_bad_inc_map: run_osd td/bad-inc-map 2 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T02:58:58.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/2 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 ' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/2' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/2/journal' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-26T02:58:58.104 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-26T02:58:58.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T02:58:58.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/2 2026-02-26T02:58:58.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T02:58:58.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bef1989e-1be7-4ff1-88b5-4211f205bb27 2026-02-26T02:58:58.107 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 bef1989e-1be7-4ff1-88b5-4211f205bb27 2026-02-26T02:58:58.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 bef1989e-1be7-4ff1-88b5-4211f205bb27' 2026-02-26T02:58:58.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T02:58:58.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDytp9piTy8BhAA7IIrsQoPiY5DbcL9yHKHKg== 2026-02-26T02:58:58.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDytp9piTy8BhAA7IIrsQoPiY5DbcL9yHKHKg=="}' 2026-02-26T02:58:58.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bef1989e-1be7-4ff1-88b5-4211f205bb27 -i td/bad-inc-map/2/new.json 2026-02-26T02:58:58.318 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:58:58.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/2/new.json 2026-02-26T02:58:58.327 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDytp9piTy8BhAA7IIrsQoPiY5DbcL9yHKHKg== --osd-uuid bef1989e-1be7-4ff1-88b5-4211f205bb27 2026-02-26T02:58:58.342 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:58.338+0000 7f360d3f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:58.344 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:58.338+0000 7f360d3f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:58.345 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:58.338+0000 7f360d3f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:58:58.345 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:58.338+0000 7f360d3f98c0 -1 bdev(0x55d9ad051c00 td/bad-inc-map/2/block) open stat got: (1) Operation not permitted 2026-02-26T02:58:58.346 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:58:58.338+0000 7f360d3f98c0 -1 bluestore(td/bad-inc-map/2) _read_fsid unparsable uuid 2026-02-26T02:59:00.611 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/2/keyring 2026-02-26T02:59:00.611 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T02:59:00.612 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T02:59:00.612 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T02:59:00.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T02:59:00.879 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T02:59:00.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T02:59:00.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=68c23698-cbf2-49ec-a58f-99ead9a6a33d --auth-supported=none --mon-host=127.0.0.1:50755 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11865/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T02:59:00.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T02:59:00.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T02:59:00.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T02:59:00.899 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:00.894+0000 7f1b8c6168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:00.901 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:00.894+0000 7f1b8c6168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:00.902 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:00.898+0000 7f1b8c6168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:01.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:01.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:02.276 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:59:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T02:59:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:02.403 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:02.398+0000 7f1b8c6168c0 -1 Falling back to public interface 2026-02-26T02:59:02.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:03.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:03.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:03.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T02:59:03.475 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:59:03.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:03.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:03.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:04.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:04.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:04.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T02:59:04.681 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T02:59:04.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:04.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:04.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:05.642 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:05.634+0000 7f1b8c6168c0 -1 osd.2 0 log_to_monitors true 2026-02-26T02:59:05.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:05.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:05.879 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T02:59:05.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T02:59:05.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:05.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:06.077 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:07.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:07.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:07.079 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T02:59:07.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T02:59:07.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:07.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:07.282 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/516009691,v1:127.0.0.1:6819/516009691] [v2:127.0.0.1:6820/516009691,v1:127.0.0.1:6821/516009691] exists,up bef1989e-1be7-4ff1-88b5-4211f205bb27 2026-02-26T02:59:07.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:07.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:07.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:07.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:34: TEST_bad_inc_map: ceph config set osd.2 osd_inject_bad_map_crc_probability 1 2026-02-26T02:59:07.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:37: TEST_bad_inc_map: create_pool foo 8 2026-02-26T02:59:07.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 8 2026-02-26T02:59:07.744 INFO:tasks.workunit.client.0.trial123.stderr:pool 'foo' already exists 2026-02-26T02:59:07.754 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T02:59:08.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:38: TEST_bad_inc_map: ceph osd pool set foo min_size 1 2026-02-26T02:59:09.058 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 min_size to 1 2026-02-26T02:59:09.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:39: TEST_bad_inc_map: ceph osd pool set foo min_size 2 2026-02-26T02:59:09.362 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 min_size to 2 2026-02-26T02:59:09.374 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:41: TEST_bad_inc_map: sleep 5 2026-02-26T02:59:14.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: TIMEOUT=10 2026-02-26T02:59:14.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: wait_for_osd up 0 2026-02-26T02:59:14.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:14.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T02:59:14.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:14.377 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:14.377 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:14.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-26T02:59:14.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:14.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:14.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:14.579 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4073897922,v1:127.0.0.1:6803/4073897922] [v2:127.0.0.1:6804/4073897922,v1:127.0.0.1:6805/4073897922] exists,up 6c3c37f7-7247-4a64-9ece-6bd413359701 2026-02-26T02:59:14.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: TIMEOUT=10 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: wait_for_osd up 1 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:14.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:14.776 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4276654763,v1:127.0.0.1:6811/4276654763] [v2:127.0.0.1:6812/4276654763,v1:127.0.0.1:6813/4276654763] exists,up f0f662ce-921a-40fd-a288-96b40b8f449d 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: TIMEOUT=10 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: wait_for_osd up 2 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:14.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-26T02:59:14.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:14.779 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:14.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:14.780 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:14.973 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/516009691,v1:127.0.0.1:6819/516009691] [v2:127.0.0.1:6820/516009691,v1:127.0.0.1:6821/516009691] exists,up bef1989e-1be7-4ff1-88b5-4211f205bb27 2026-02-26T02:59:14.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:14.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:14.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:14.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:49: TEST_bad_inc_map: grep 'injecting map crc failure' td/bad-inc-map/osd.2.log 2026-02-26T02:59:14.975 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.694+0000 7f1b7bc6f640 -1 osd.2 13 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.975 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.694+0000 7f1b79c6b640 -1 osd.2 13 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.975 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.794+0000 7f1b7bc6f640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.975 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.794+0000 7f1b7bc6f640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.975 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:08.494+0000 7f1b79c6b640 -1 osd.2 15 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.976 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:08.950+0000 7f1b7bc6f640 -1 osd.2 16 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.976 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.054+0000 7f1b7bc6f640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.976 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.054+0000 7f1b79c6b640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.976 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.254+0000 7f1b7bc6f640 -1 osd.2 18 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.976 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.358+0000 7f1b7bc6f640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.976 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.358+0000 7f1b79c6b640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-26T02:59:14.977 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:50: TEST_bad_inc_map: grep 'bailing because last' td/bad-inc-map/osd.2.log 2026-02-26T02:59:14.977 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.694+0000 7f1b7bc6f640 10 osd.2 13 handle_osd_map bailing because last < start (13<14) 2026-02-26T02:59:14.977 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.694+0000 7f1b79c6b640 10 osd.2 13 handle_osd_map bailing because last < start (13<14) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.794+0000 7f1b7bc6f640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:07.794+0000 7f1b7bc6f640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:08.494+0000 7f1b79c6b640 10 osd.2 15 handle_osd_map bailing because last < start (15<16) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:08.950+0000 7f1b7bc6f640 10 osd.2 16 handle_osd_map bailing because last < start (16<17) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.054+0000 7f1b7bc6f640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.054+0000 7f1b79c6b640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-26T02:59:14.978 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.254+0000 7f1b7bc6f640 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2026-02-26T02:59:14.979 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.358+0000 7f1b7bc6f640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-26T02:59:14.979 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T02:59:09.358+0000 7f1b79c6b640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-26T02:59:14.979 INFO:tasks.workunit.client.0.trial123.stdout:success 2026-02-26T02:59:14.979 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:52: TEST_bad_inc_map: echo success 2026-02-26T02:59:14.980 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:54: TEST_bad_inc_map: delete_pool foo 2026-02-26T02:59:14.980 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=foo 2026-02-26T02:59:14.980 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete foo foo --yes-i-really-really-mean-it 2026-02-26T02:59:15.218 INFO:tasks.workunit.client.0.trial123.stderr:pool 'foo' does not exist 2026-02-26T02:59:15.227 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:55: TEST_bad_inc_map: kill_daemons td/bad-inc-map 2026-02-26T02:59:15.228 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T02:59:15.228 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T02:59:15.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T02:59:15.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T02:59:15.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T02:59:23.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T02:59:23.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:21: run: teardown td/bad-inc-map 2026-02-26T02:59:23.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-26T02:59:23.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T02:59:23.546 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-26T02:59:23.546 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T02:59:23.546 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T02:59:23.546 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T02:59:23.546 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T02:59:23.546 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T02:59:23.551 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T02:59:23.551 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T02:59:23.552 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T02:59:23.552 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T02:59:23.553 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T02:59:23.553 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T02:59:23.554 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T02:59:23.556 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:59:23.556 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T02:59:23.556 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T02:59:23.557 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:59:23.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T02:59:23.558 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T02:59:23.558 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-26T02:59:23.569 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T02:59:23.569 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.569 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:59:23.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11865 2026-02-26T02:59:23.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T02:59:23.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/bad-inc-map 0 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T02:59:23.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T02:59:23.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T02:59:23.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T02:59:23.574 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T02:59:23.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T02:59:23.575 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T02:59:23.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T02:59:23.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T02:59:23.576 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T02:59:23.577 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:59:23.577 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T02:59:23.577 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T02:59:23.577 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:59:23.578 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T02:59:23.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-26T02:59:23.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-26T02:59:23.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T02:59:23.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11865 2026-02-26T02:59:23.581 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11865 2026-02-26T02:59:23.581 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T02:59:23.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T02:59:23.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-26T02:59:23.583 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-26T02:59:23.583 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T02:59:23.634 INFO:tasks.workunit:Running workunit osd/divergent-priors.sh... 2026-02-26T02:59:23.635 DEBUG:teuthology.orchestra.run.trial123:workunit test osd/divergent-priors.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh 2026-02-26T02:59:23.687 INFO:tasks.workunit.client.0.trial123.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-26T02:59:23.690 INFO:tasks.workunit.client.0.trial123.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/divergent-priors 2026-02-26T02:59:23.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:21: run: local dir=td/divergent-priors 2026-02-26T02:59:23.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:22: run: shift 2026-02-26T02:59:23.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: export loglen=12 2026-02-26T02:59:23.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: loglen=12 2026-02-26T02:59:23.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: export divisor=3 2026-02-26T02:59:23.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: divisor=3 2026-02-26T02:59:23.692 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: expr 12 / 2 2026-02-26T02:59:23.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: export trim=6 2026-02-26T02:59:23.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: trim=6 2026-02-26T02:59:23.693 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: expr 6 / 3 2026-02-26T02:59:23.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: export DIVERGENT_WRITE=2 2026-02-26T02:59:23.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: DIVERGENT_WRITE=2 2026-02-26T02:59:23.694 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: expr 6 / 3 2026-02-26T02:59:23.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: export DIVERGENT_REMOVE=2 2026-02-26T02:59:23.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: DIVERGENT_REMOVE=2 2026-02-26T02:59:23.695 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: expr 6 / 3 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: export DIVERGENT_CREATE=2 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: DIVERGENT_CREATE=2 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: export poolname=test 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: poolname=test 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: export testobjects=100 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: testobjects=100 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: export CEPH_MON=127.0.0.1:7115 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: CEPH_MON=127.0.0.1:7115 2026-02-26T02:59:23.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:35: run: export CEPH_ARGS 2026-02-26T02:59:23.698 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: uuidgen 2026-02-26T02:59:23.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: CEPH_ARGS+='--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none ' 2026-02-26T02:59:23.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:37: run: CEPH_ARGS+='--mon-host=127.0.0.1:7115 ' 2026-02-26T02:59:23.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:39: run: CEPH_ARGS+='--osd_force_auth_primary_missing_objects=1000000 ' 2026-02-26T02:59:23.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:40: run: CEPH_ARGS+='--osd_debug_pg_log_writeout=true ' 2026-02-26T02:59:23.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:41: run: CEPH_ARGS+='--osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T02:59:23.699 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: set 2026-02-26T02:59:23.699 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: local 'funcs=TEST_divergent 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:TEST_divergent_2 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:TEST_divergent_3 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:TEST_divergent_ec' 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-26T02:59:23.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T02:59:23.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T02:59:23.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T02:59:23.704 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T02:59:23.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T02:59:23.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T02:59:23.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T02:59:23.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T02:59:23.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T02:59:23.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:59:23.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T02:59:23.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T02:59:23.708 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T02:59:23.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T02:59:23.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T02:59:23.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T02:59:23.711 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T02:59:23.712 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.712 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:23.712 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T02:59:23.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T02:59:23.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T02:59:23.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-26T02:59:23.715 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T02:59:23.715 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.715 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:23.715 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.18886 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent td/divergent-priors 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:57: TEST_divergent: local dir=td/divergent-priors 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:60: TEST_divergent: local dummyfile=/etc/fstab 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: local dummyfile2=/etc/resolv.conf 2026-02-26T02:59:23.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:63: TEST_divergent: local num_osds=3 2026-02-26T02:59:23.718 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: expr 3 - 1 2026-02-26T02:59:23.719 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: seq 0 2 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: local 'osds=0 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:65: TEST_divergent: run_mon td/divergent-priors a 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-26T02:59:23.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-26T02:59:23.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T02:59:23.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:59:23.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:59:23.758 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:59:23.758 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.758 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:23.758 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:23.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T02:59:23.791 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T02:59:23.792 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T02:59:23.792 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T02:59:23.792 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T02:59:23.792 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T02:59:23.792 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T02:59:23.792 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T02:59:23.793 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T02:59:23.793 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T02:59:23.793 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T02:59:23.793 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.793 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:23.793 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T02:59:23.794 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T02:59:23.794 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get fsid 2026-02-26T02:59:23.849 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T02:59:23.850 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:23.851 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:23.851 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T02:59:23.851 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T02:59:23.851 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get mon_host 2026-02-26T02:59:23.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_mgr td/divergent-priors x 2026-02-26T02:59:23.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-26T02:59:23.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T02:59:23.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T02:59:23.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T02:59:23.905 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-26T02:59:23.905 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T02:59:24.016 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T02:59:24.017 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:59:24.017 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:59:24.017 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:59:24.017 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:24.017 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:24.017 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:24.018 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T02:59:24.018 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T02:59:24.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-26T02:59:24.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 0 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T02:59:24.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:24.042 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T02:59:24.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T02:59:24.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T02:59:24.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T02:59:24.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-26T02:59:24.044 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T02:59:24.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=93440a18-03ce-414c-9db5-dcd76945e44d 2026-02-26T02:59:24.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 93440a18-03ce-414c-9db5-dcd76945e44d' 2026-02-26T02:59:24.045 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 93440a18-03ce-414c-9db5-dcd76945e44d 2026-02-26T02:59:24.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T02:59:24.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAMt59p0ODzAhAANQFs3c851wLp4a10YNkySA== 2026-02-26T02:59:24.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAMt59p0ODzAhAANQFs3c851wLp4a10YNkySA=="}' 2026-02-26T02:59:24.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 93440a18-03ce-414c-9db5-dcd76945e44d -i td/divergent-priors/0/new.json 2026-02-26T02:59:24.164 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:24.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-26T02:59:24.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMt59p0ODzAhAANQFs3c851wLp4a10YNkySA== --osd-uuid 93440a18-03ce-414c-9db5-dcd76945e44d 2026-02-26T02:59:24.190 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:24.186+0000 7fb06abac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:24.193 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:24.186+0000 7fb06abac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:24.193 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:24.186+0000 7fb06abac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:24.194 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:24.190+0000 7fb06abac8c0 -1 bdev(0x5603622f1c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-26T02:59:24.194 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:24.190+0000 7fb06abac8c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-26T02:59:26.782 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-26T02:59:26.782 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T02:59:26.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T02:59:26.783 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T02:59:26.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T02:59:27.067 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T02:59:27.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T02:59:27.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T02:59:27.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T02:59:27.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T02:59:27.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T02:59:27.086 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:27.082+0000 7f35c43888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:27.087 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:27.082+0000 7f35c43888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:27.088 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:27.082+0000 7f35c43888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:27.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T02:59:27.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:27.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T02:59:27.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:27.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:27.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:27.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:27.269 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:27.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:27.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:27.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:28.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:28.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:28.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T02:59:28.467 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:59:28.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:28.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:28.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:28.816 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:28.810+0000 7f35c43888c0 -1 Falling back to public interface 2026-02-26T02:59:29.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:29.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:29.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T02:59:29.666 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:59:29.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:29.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:29.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:30.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:30.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:30.862 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T02:59:30.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T02:59:30.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:30.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:31.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:31.805 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:31.798+0000 7f35c43888c0 -1 osd.0 0 log_to_monitors true 2026-02-26T02:59:32.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:32.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:32.066 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T02:59:32.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T02:59:32.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:32.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:32.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:33.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:33.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:33.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T02:59:33.271 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T02:59:33.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:33.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T02:59:33.469 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/538168964,v1:127.0.0.1:6803/538168964] [v2:127.0.0.1:6804/538168964,v1:127.0.0.1:6805/538168964] exists,up 93440a18-03ce-414c-9db5-dcd76945e44d 2026-02-26T02:59:33.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:33.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:33.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:33.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-26T02:59:33.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 1 2026-02-26T02:59:33.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T02:59:33.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-26T02:59:33.473 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T02:59:33.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=434d09e9-f550-42dd-bd63-e6b534a21fde 2026-02-26T02:59:33.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 434d09e9-f550-42dd-bd63-e6b534a21fde' 2026-02-26T02:59:33.473 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 434d09e9-f550-42dd-bd63-e6b534a21fde 2026-02-26T02:59:33.473 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T02:59:33.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAVt59pD6r2HBAAxuJ/XaiHPwFurrQMAI6FrQ== 2026-02-26T02:59:33.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAVt59pD6r2HBAAxuJ/XaiHPwFurrQMAI6FrQ=="}' 2026-02-26T02:59:33.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 434d09e9-f550-42dd-bd63-e6b534a21fde -i td/divergent-priors/1/new.json 2026-02-26T02:59:33.700 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:59:33.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-26T02:59:33.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAVt59pD6r2HBAAxuJ/XaiHPwFurrQMAI6FrQ== --osd-uuid 434d09e9-f550-42dd-bd63-e6b534a21fde 2026-02-26T02:59:33.729 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:33.722+0000 7f1962fbb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:33.730 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:33.726+0000 7f1962fbb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:33.732 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:33.726+0000 7f1962fbb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:33.732 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:33.726+0000 7f1962fbb8c0 -1 bdev(0x55807da27c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-26T02:59:33.732 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:33.726+0000 7f1962fbb8c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-26T02:59:35.998 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-26T02:59:35.998 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T02:59:35.999 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T02:59:35.999 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T02:59:36.000 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T02:59:36.265 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T02:59:36.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T02:59:36.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T02:59:36.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T02:59:36.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T02:59:36.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T02:59:36.294 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:36.290+0000 7faa1201b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:36.296 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:36.290+0000 7faa1201b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:36.297 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:36.290+0000 7faa1201b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:36.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T02:59:36.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:36.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T02:59:36.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:36.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:36.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:36.468 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:36.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:36.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:36.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:36.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:37.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:37.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:37.668 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:59:37.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T02:59:37.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:37.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:37.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:38.020 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:38.014+0000 7faa1201b8c0 -1 Falling back to public interface 2026-02-26T02:59:38.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:38.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:38.864 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:59:38.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T02:59:38.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:38.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:39.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:40.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:40.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:40.066 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T02:59:40.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T02:59:40.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:40.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:40.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:40.998 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:40.993+0000 7faa1201b8c0 -1 osd.1 0 log_to_monitors true 2026-02-26T02:59:41.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:41.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:41.263 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T02:59:41.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T02:59:41.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:41.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:41.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:42.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:42.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:42.467 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T02:59:42.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T02:59:42.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:42.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T02:59:42.665 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3458897790,v1:127.0.0.1:6811/3458897790] [v2:127.0.0.1:6812/3458897790,v1:127.0.0.1:6813/3458897790] exists,up 434d09e9-f550-42dd-bd63-e6b534a21fde 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 2 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T02:59:42.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T02:59:42.668 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T02:59:42.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-26T02:59:42.670 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T02:59:42.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b647e66c-bee7-46bc-a1fb-3f64d96c08d0 2026-02-26T02:59:42.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 b647e66c-bee7-46bc-a1fb-3f64d96c08d0' 2026-02-26T02:59:42.671 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 b647e66c-bee7-46bc-a1fb-3f64d96c08d0 2026-02-26T02:59:42.672 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T02:59:42.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAet59pVrmxKBAA4LrF/6DGq6WtaAF+Twv3Ug== 2026-02-26T02:59:42.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAet59pVrmxKBAA4LrF/6DGq6WtaAF+Twv3Ug=="}' 2026-02-26T02:59:42.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b647e66c-bee7-46bc-a1fb-3f64d96c08d0 -i td/divergent-priors/2/new.json 2026-02-26T02:59:42.897 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:59:42.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-26T02:59:42.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAet59pVrmxKBAA4LrF/6DGq6WtaAF+Twv3Ug== --osd-uuid b647e66c-bee7-46bc-a1fb-3f64d96c08d0 2026-02-26T02:59:42.934 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:42.929+0000 7ff6f01f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:42.936 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:42.929+0000 7ff6f01f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:42.937 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:42.929+0000 7ff6f01f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:42.938 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:42.929+0000 7ff6f01f38c0 -1 bdev(0x55ed97a4fc00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-26T02:59:42.938 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:42.929+0000 7ff6f01f38c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-26T02:59:45.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-26T02:59:45.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T02:59:45.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T02:59:45.208 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T02:59:45.209 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T02:59:45.478 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T02:59:45.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T02:59:45.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T02:59:45.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T02:59:45.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T02:59:45.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T02:59:45.497 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:45.489+0000 7fd169a738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:45.499 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:45.493+0000 7fd169a738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:45.500 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:45.493+0000 7fd169a738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T02:59:45.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T02:59:45.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T02:59:45.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T02:59:45.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T02:59:45.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T02:59:45.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:45.677 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T02:59:45.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T02:59:45.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:45.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:45.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:46.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:46.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:46.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T02:59:46.877 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T02:59:46.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:46.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:47.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:47.227 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:47.221+0000 7fd169a738c0 -1 Falling back to public interface 2026-02-26T02:59:48.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:48.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:48.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T02:59:48.080 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T02:59:48.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:48.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:48.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:49.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:49.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:49.283 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T02:59:49.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T02:59:49.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:49.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:49.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:50.246 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T02:59:50.241+0000 7fd169a738c0 -1 osd.2 0 log_to_monitors true 2026-02-26T02:59:50.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:50.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:50.486 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T02:59:50.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T02:59:50.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:50.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:50.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T02:59:51.690 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T02:59:51.690 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T02:59:51.690 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T02:59:51.691 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T02:59:51.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T02:59:51.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T02:59:51.899 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/585149031,v1:127.0.0.1:6819/585149031] [v2:127.0.0.1:6820/585149031,v1:127.0.0.1:6821/585149031] exists,up b647e66c-bee7-46bc-a1fb-3f64d96c08d0 2026-02-26T02:59:51.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T02:59:51.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T02:59:51.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T02:59:51.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:72: TEST_divergent: ceph osd set noout 2026-02-26T02:59:52.201 INFO:tasks.workunit.client.0.trial123.stderr:noout is set 2026-02-26T02:59:52.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:73: TEST_divergent: ceph osd set noin 2026-02-26T02:59:52.507 INFO:tasks.workunit.client.0.trial123.stderr:noin is set 2026-02-26T02:59:52.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:74: TEST_divergent: ceph osd set nodown 2026-02-26T02:59:52.810 INFO:tasks.workunit.client.0.trial123.stderr:nodown is set 2026-02-26T02:59:52.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:75: TEST_divergent: create_pool test 1 1 2026-02-26T02:59:52.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-26T02:59:53.060 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' already exists 2026-02-26T02:59:53.071 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T02:59:54.072 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:76: TEST_divergent: ceph osd pool set test size 3 2026-02-26T02:59:54.412 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 size to 3 2026-02-26T02:59:54.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:77: TEST_divergent: ceph osd pool set test min_size 2 2026-02-26T02:59:54.718 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 min_size to 2 2026-02-26T02:59:54.729 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:79: TEST_divergent: flush_pg_stats 2026-02-26T02:59:54.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T02:59:54.730 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T02:59:54.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T02:59:54.926 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T02:59:54.926 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T02:59:54.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T02:59:54.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T02:59:54.926 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T02:59:55.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-26T02:59:55.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-26T02:59:55.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-26T02:59:55.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T02:59:55.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T02:59:55.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-26T02:59:55.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-26T02:59:55.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-26T02:59:55.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T02:59:55.087 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T02:59:55.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-26T02:59:55.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-26T02:59:55.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852' 2026-02-26T02:59:55.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T02:59:55.167 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-26T02:59:55.167 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T02:59:55.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T02:59:55.170 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-26T02:59:55.170 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T02:59:55.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-26T02:59:55.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-26T02:59:55.171 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836487 2026-02-26T02:59:55.172 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T02:59:55.379 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-26T02:59:55.379 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T02:59:56.379 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T02:59:56.380 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T02:59:56.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-26T02:59:56.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T02:59:56.583 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-26T02:59:56.583 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T02:59:56.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T02:59:56.585 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-26T02:59:56.585 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T02:59:56.587 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-26T02:59:56.587 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-26T02:59:56.587 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 38654705669 2026-02-26T02:59:56.587 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T02:59:56.792 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-26T02:59:56.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T02:59:56.793 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-26T02:59:56.793 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T02:59:56.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T02:59:56.795 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-26T02:59:56.795 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T02:59:56.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-26T02:59:56.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-26T02:59:56.797 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 55834574852 2026-02-26T02:59:56.797 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T02:59:57.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-26T02:59:57.013 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: wait_for_clean 2026-02-26T02:59:57.013 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T02:59:57.013 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T02:59:57.013 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T02:59:57.014 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T02:59:57.014 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T02:59:57.014 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T02:59:57.014 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T02:59:57.014 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T02:59:57.014 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T02:59:57.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T02:59:57.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T02:59:57.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T02:59:57.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T02:59:57.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T02:59:57.109 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T02:59:57.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T02:59:57.312 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T02:59:57.312 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T02:59:57.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T02:59:57.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T02:59:57.312 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T02:59:57.391 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-26T02:59:57.391 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-26T02:59:57.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-26T02:59:57.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T02:59:57.392 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T02:59:57.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-26T02:59:57.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-26T02:59:57.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-26T02:59:57.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T02:59:57.472 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T02:59:57.551 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574855 2026-02-26T02:59:57.551 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574855 2026-02-26T02:59:57.551 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855' 2026-02-26T02:59:57.551 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T02:59:57.552 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-26T02:59:57.552 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T02:59:57.552 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T02:59:57.553 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-26T02:59:57.553 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T02:59:57.555 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-26T02:59:57.555 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-26T02:59:57.555 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836489 2026-02-26T02:59:57.555 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T02:59:57.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-26T02:59:57.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T02:59:58.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T02:59:58.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T02:59:58.963 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-26T02:59:58.963 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T02:59:58.963 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-26T02:59:58.964 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T02:59:58.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T02:59:58.966 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-26T02:59:58.966 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T02:59:58.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-26T02:59:58.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-26T02:59:58.968 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 38654705672 2026-02-26T02:59:58.968 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T02:59:59.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-26T02:59:59.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T02:59:59.166 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574855 2026-02-26T02:59:59.166 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T02:59:59.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T02:59:59.169 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574855 2026-02-26T02:59:59.169 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T02:59:59.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574855 2026-02-26T02:59:59.170 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 55834574855 2026-02-26T02:59:59.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-26T02:59:59.171 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T02:59:59.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-26T02:59:59.366 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T02:59:59.366 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T02:59:59.366 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T02:59:59.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T02:59:59.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T02:59:59.632 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T02:59:59.632 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T02:59:59.632 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T02:59:59.632 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T02:59:59.632 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T02:59:59.633 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T02:59:59.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T02:59:59.834 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T02:59:59.835 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T02:59:59.835 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:00.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:00:00.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:00:00.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:00:00.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: ceph pg dump pgs --format=json 2026-02-26T03:00:00.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: jq '.pg_stats[0].up_primary' 2026-02-26T03:00:00.293 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:00:00.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: local divergent=1 2026-02-26T03:00:00.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: echo 'primary and soon to be divergent is 1' 2026-02-26T03:00:00.303 INFO:tasks.workunit.client.0.trial123.stdout:primary and soon to be divergent is 1 2026-02-26T03:00:00.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:85: TEST_divergent: ceph pg dump pgs 2026-02-26T03:00:00.491 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:00:00.491 INFO:tasks.workunit.client.0.trial123.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-26T02:59:53.246436+0000 0'0 26:21 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T05:03:35.663686+0000 0 0 2026-02-26T03:00:00.491 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:00:00.492 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:00:00.492 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:00:00.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: local non_divergent= 2026-02-26T03:00:00.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-26T03:00:00.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 0 = 1 ']' 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0' 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 1 = 1 ']' 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:90: TEST_divergent: continue 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 2 = 1 ']' 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0 2' 2026-02-26T03:00:00.501 INFO:tasks.workunit.client.0.trial123.stdout:writing initial objects 2026-02-26T03:00:00.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:95: TEST_divergent: echo 'writing initial objects' 2026-02-26T03:00:00.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: seq 1 100 2026-02-26T03:00:00.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_1 /etc/fstab 2026-02-26T03:00:00.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_2 /etc/fstab 2026-02-26T03:00:00.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_3 /etc/fstab 2026-02-26T03:00:00.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_4 /etc/fstab 2026-02-26T03:00:00.595 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.595 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_5 /etc/fstab 2026-02-26T03:00:00.614 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.614 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_6 /etc/fstab 2026-02-26T03:00:00.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_7 /etc/fstab 2026-02-26T03:00:00.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_8 /etc/fstab 2026-02-26T03:00:00.690 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_9 /etc/fstab 2026-02-26T03:00:00.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.719 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_10 /etc/fstab 2026-02-26T03:00:00.745 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.745 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_11 /etc/fstab 2026-02-26T03:00:00.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_12 /etc/fstab 2026-02-26T03:00:00.803 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.803 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_13 /etc/fstab 2026-02-26T03:00:00.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_14 /etc/fstab 2026-02-26T03:00:00.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_15 /etc/fstab 2026-02-26T03:00:00.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_16 /etc/fstab 2026-02-26T03:00:00.914 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_17 /etc/fstab 2026-02-26T03:00:00.943 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.943 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_18 /etc/fstab 2026-02-26T03:00:00.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_19 /etc/fstab 2026-02-26T03:00:00.998 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:00.998 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_20 /etc/fstab 2026-02-26T03:00:01.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_21 /etc/fstab 2026-02-26T03:00:01.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_22 /etc/fstab 2026-02-26T03:00:01.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_23 /etc/fstab 2026-02-26T03:00:01.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_24 /etc/fstab 2026-02-26T03:00:01.134 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.134 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_25 /etc/fstab 2026-02-26T03:00:01.162 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.162 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_26 /etc/fstab 2026-02-26T03:00:01.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_27 /etc/fstab 2026-02-26T03:00:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_28 /etc/fstab 2026-02-26T03:00:01.232 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.232 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_29 /etc/fstab 2026-02-26T03:00:01.260 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.260 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_30 /etc/fstab 2026-02-26T03:00:01.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_31 /etc/fstab 2026-02-26T03:00:01.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_32 /etc/fstab 2026-02-26T03:00:01.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_33 /etc/fstab 2026-02-26T03:00:01.360 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.360 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_34 /etc/fstab 2026-02-26T03:00:01.387 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.387 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_35 /etc/fstab 2026-02-26T03:00:01.415 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.415 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_36 /etc/fstab 2026-02-26T03:00:01.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_37 /etc/fstab 2026-02-26T03:00:01.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_38 /etc/fstab 2026-02-26T03:00:01.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_39 /etc/fstab 2026-02-26T03:00:01.526 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.526 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_40 /etc/fstab 2026-02-26T03:00:01.554 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.554 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_41 /etc/fstab 2026-02-26T03:00:01.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_42 /etc/fstab 2026-02-26T03:00:01.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_43 /etc/fstab 2026-02-26T03:00:01.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_44 /etc/fstab 2026-02-26T03:00:01.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_45 /etc/fstab 2026-02-26T03:00:01.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_46 /etc/fstab 2026-02-26T03:00:01.715 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.715 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_47 /etc/fstab 2026-02-26T03:00:01.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_48 /etc/fstab 2026-02-26T03:00:01.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_49 /etc/fstab 2026-02-26T03:00:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_50 /etc/fstab 2026-02-26T03:00:01.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_51 /etc/fstab 2026-02-26T03:00:01.851 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.852 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_52 /etc/fstab 2026-02-26T03:00:01.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_53 /etc/fstab 2026-02-26T03:00:01.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_54 /etc/fstab 2026-02-26T03:00:01.934 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_55 /etc/fstab 2026-02-26T03:00:01.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_56 /etc/fstab 2026-02-26T03:00:01.990 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:01.990 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_57 /etc/fstab 2026-02-26T03:00:02.018 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.018 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_58 /etc/fstab 2026-02-26T03:00:02.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_59 /etc/fstab 2026-02-26T03:00:02.074 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.074 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_60 /etc/fstab 2026-02-26T03:00:02.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_61 /etc/fstab 2026-02-26T03:00:02.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_62 /etc/fstab 2026-02-26T03:00:02.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_63 /etc/fstab 2026-02-26T03:00:02.184 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.184 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_64 /etc/fstab 2026-02-26T03:00:02.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_65 /etc/fstab 2026-02-26T03:00:02.230 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_66 /etc/fstab 2026-02-26T03:00:02.259 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.259 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_67 /etc/fstab 2026-02-26T03:00:02.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_68 /etc/fstab 2026-02-26T03:00:02.314 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_69 /etc/fstab 2026-02-26T03:00:02.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_70 /etc/fstab 2026-02-26T03:00:02.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_71 /etc/fstab 2026-02-26T03:00:02.398 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.398 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_72 /etc/fstab 2026-02-26T03:00:02.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.427 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_73 /etc/fstab 2026-02-26T03:00:02.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_74 /etc/fstab 2026-02-26T03:00:02.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_75 /etc/fstab 2026-02-26T03:00:02.510 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.510 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_76 /etc/fstab 2026-02-26T03:00:02.538 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.538 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_77 /etc/fstab 2026-02-26T03:00:02.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_78 /etc/fstab 2026-02-26T03:00:02.594 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.594 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_79 /etc/fstab 2026-02-26T03:00:02.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.622 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_80 /etc/fstab 2026-02-26T03:00:02.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_81 /etc/fstab 2026-02-26T03:00:02.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_82 /etc/fstab 2026-02-26T03:00:02.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_83 /etc/fstab 2026-02-26T03:00:02.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_84 /etc/fstab 2026-02-26T03:00:02.732 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.732 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_85 /etc/fstab 2026-02-26T03:00:02.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.757 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_86 /etc/fstab 2026-02-26T03:00:02.776 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.776 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_87 /etc/fstab 2026-02-26T03:00:02.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_88 /etc/fstab 2026-02-26T03:00:02.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_89 /etc/fstab 2026-02-26T03:00:02.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_90 /etc/fstab 2026-02-26T03:00:02.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_91 /etc/fstab 2026-02-26T03:00:02.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_92 /etc/fstab 2026-02-26T03:00:02.944 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.944 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_93 /etc/fstab 2026-02-26T03:00:02.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:02.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_94 /etc/fstab 2026-02-26T03:00:03.000 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:03.000 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_95 /etc/fstab 2026-02-26T03:00:03.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:03.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_96 /etc/fstab 2026-02-26T03:00:03.055 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:03.055 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_97 /etc/fstab 2026-02-26T03:00:03.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:03.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_98 /etc/fstab 2026-02-26T03:00:03.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:03.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_99 /etc/fstab 2026-02-26T03:00:03.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-26T03:00:03.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_100 /etc/fstab 2026-02-26T03:00:03.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:00:03.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: wait_for_clean 2026-02-26T03:00:03.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:00:03.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:00:03.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:00:03.158 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:00:03.159 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:00:03.159 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:00:03.159 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:00:03.159 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:00:03.159 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:00:03.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:00:03.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:00:03.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:00:03.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:00:03.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:00:03.229 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:00:03.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:00:03.428 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:00:03.428 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:00:03.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:00:03.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:00:03.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:00:03.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-26T03:00:03.508 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-26T03:00:03.508 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-26T03:00:03.508 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:00:03.508 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:00:03.589 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-26T03:00:03.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-26T03:00:03.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675' 2026-02-26T03:00:03.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:00:03.590 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:00:03.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574858 2026-02-26T03:00:03.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574858 2026-02-26T03:00:03.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675 2-55834574858' 2026-02-26T03:00:03.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:00:03.667 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-26T03:00:03.667 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:00:03.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:00:03.669 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-26T03:00:03.669 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:00:03.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-26T03:00:03.670 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836493 2026-02-26T03:00:03.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-26T03:00:03.671 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:00:03.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836493 2026-02-26T03:00:03.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:00:04.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:00:04.888 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:00:05.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-26T03:00:05.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:00:05.087 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-26T03:00:05.087 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:00:05.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:00:05.089 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-26T03:00:05.089 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:00:05.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-26T03:00:05.090 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 38654705675 2026-02-26T03:00:05.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-26T03:00:05.091 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:00:05.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-26T03:00:05.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:00:05.312 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574858 2026-02-26T03:00:05.312 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:00:05.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:00:05.314 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574858 2026-02-26T03:00:05.314 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:00:05.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574858 2026-02-26T03:00:05.316 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 55834574858 2026-02-26T03:00:05.317 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574858' 2026-02-26T03:00:05.317 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:00:05.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-26T03:00:05.529 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:00:05.529 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:05.529 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:05.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:00:05.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:05.797 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:05.797 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:05.797 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:05.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:05.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:05.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:05.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:00:05.995 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:05.995 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:05.995 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:06.258 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:00:06.258 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:00:06.258 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:00:06.259 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: get_pg test existing_1 2026-02-26T03:00:06.259 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-26T03:00:06.260 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-26T03:00:06.260 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-26T03:00:06.260 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-26T03:00:06.461 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: local pgid=1.0 2026-02-26T03:00:06.462 INFO:tasks.workunit.client.0.trial123.stdout:blackholing osds 0 2 2026-02-26T03:00:06.462 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:107: TEST_divergent: echo 'blackholing osds 0 2' 2026-02-26T03:00:06.462 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph pg dump pgs 2026-02-26T03:00:06.647 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:00:06.647 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-26T02:59:53.246436+0000 29'100 29:125 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T05:03:35.663686+0000 0 0 2026-02-26T03:00:06.647 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:00:06.647 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:00:06.648 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:00:06.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:06.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.0 2026-02-26T03:00:06.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:00:06.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:00:06.656 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:00:06.656 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:00:06.656 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:00:06.656 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:00:06.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2026-02-26T03:00:06.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-26T03:00:06.711 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:00:06.711 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:00:06.711 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:00:06.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:06.717 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.2 2026-02-26T03:00:06.717 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:00:06.717 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:00:06.717 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:00:06.717 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:00:06.718 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:00:06.718 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:00:06.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2026-02-26T03:00:06.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-26T03:00:06.770 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:00:06.770 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:00:06.770 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:00:06.776 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:114: TEST_divergent: local case5=100 2026-02-26T03:00:06.777 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: expr 100 - 1 2026-02-26T03:00:06.777 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: local case3=99 2026-02-26T03:00:06.778 INFO:tasks.workunit.client.0.trial123.stdout:writing divergent object 2026-02-26T03:00:06.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:117: TEST_divergent: echo 'writing divergent object' 2026-02-26T03:00:06.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:119: TEST_divergent: echo 'create missing divergent object' 2026-02-26T03:00:06.780 INFO:tasks.workunit.client.0.trial123.stdout:create missing divergent object 2026-02-26T03:00:06.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:120: TEST_divergent: inject_eio rep data test existing_99 td/divergent-priors 0 2026-02-26T03:00:06.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-26T03:00:06.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-26T03:00:06.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-26T03:00:06.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-26T03:00:06.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=test 2026-02-26T03:00:06.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-26T03:00:06.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:118: TEST_divergent: rados -p test put existing_100 /etc/fstab 2026-02-26T03:00:06.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=existing_99 2026-02-26T03:00:06.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-26T03:00:06.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/divergent-priors 2026-02-26T03:00:06.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-26T03:00:06.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-26T03:00:06.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-26T03:00:06.786 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds test existing_99 2026-02-26T03:00:06.786 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=test 2026-02-26T03:00:06.786 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=existing_99 2026-02-26T03:00:06.786 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map test existing_99 2026-02-26T03:00:06.786 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-26T03:00:06.982 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-26T03:00:06.982 INFO:tasks.workunit.client.0.trial123.stderr:0 2026-02-26T03:00:06.983 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:00:06.983 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 2026-02-26T03:00:06.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0' '2') 2026-02-26T03:00:06.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-26T03:00:06.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-26T03:00:06.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-26T03:00:06.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-26T03:00:06.984 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/divergent-priors/1/type 2026-02-26T03:00:06.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-26T03:00:06.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-26T03:00:06.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-26T03:00:06.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-26T03:00:06.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-26T03:00:06.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-26T03:00:06.985 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-26T03:00:06.986 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-26T03:00:06.986 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:00:06.986 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:00:06.986 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:00:06.986 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:00:06.986 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:00:06.987 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:00:06.987 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.18886/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-26T03:00:07.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-26T03:00:07.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-26T03:00:07.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:00:07.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:00:07.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-26T03:00:07.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok injectdataerr test existing_99 2026-02-26T03:00:07.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:122: TEST_divergent: sleep 10 2026-02-26T03:00:07.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:121: TEST_divergent: rados -p test get existing_99 td/divergent-priors/existing 2026-02-26T03:00:17.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:123: TEST_divergent: killall -9 rados 2026-02-26T03:00:17.144 INFO:tasks.workunit.client.0.trial123.stderr:rados: no process found 2026-02-26T03:00:17.145 INFO:tasks.workunit.client.0.trial123.stdout:killing all the osds 2026-02-26T03:00:17.145 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: echo 'killing all the osds' 2026-02-26T03:00:17.145 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:127: TEST_divergent: ceph pg dump pgs 2026-02-26T03:00:17.348 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:00:17.348 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 17 84 17 active+clean 2026-02-26T03:00:07.122458+0000 29'101 29:134 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T05:03:35.663686+0000 0 0 2026-02-26T03:00:17.348 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:00:17.348 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:00:17.349 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:00:17.358 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: kill_daemons td/divergent-priors KILL osd 2026-02-26T03:00:17.358 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:00:17.358 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:00:17.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:00:17.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:00:17.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:00:17.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:00:17.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-26T03:00:17.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.0 2026-02-26T03:00:17.724 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:00:17.734 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-26T03:00:17.734 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.1 2026-02-26T03:00:17.996 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:00:18.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-26T03:00:18.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.2 2026-02-26T03:00:18.278 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:00:18.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:18.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.0 2026-02-26T03:00:18.555 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already out. 2026-02-26T03:00:18.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:18.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.2 2026-02-26T03:00:18.826 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already out. 2026-02-26T03:00:18.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:139: TEST_divergent: echo 'bringing up non_divergent 0 2' 2026-02-26T03:00:18.837 INFO:tasks.workunit.client.0.trial123.stdout:bringing up non_divergent 0 2 2026-02-26T03:00:18.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: ceph pg dump pgs 2026-02-26T03:00:19.022 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:00:19.023 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 17 84 17 stale+active+clean 2026-02-26T03:00:07.122458+0000 29'101 29:134 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T05:03:35.663686+0000 0 0 2026-02-26T03:00:19.023 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:00:19.023 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:00:19.023 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:00:19.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:19.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 0 2026-02-26T03:00:19.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:00:19.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:00:19.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:00:19.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:00:19.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:00:19.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:00:19.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:00:19.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:00:19.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:00:19.038 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:00:19.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:00:19.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:00:19.039 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-26T03:00:19.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:00:19.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:00:19.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:00:19.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:00:19.057 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:19.049+0000 7fbdcf3a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:00:19.058 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:19.053+0000 7fbdcf3a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:00:19.059 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:19.053+0000 7fbdcf3a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:00:19.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:00:19.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:00:19.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:00:19.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:00:19.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:00:19.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:19.236 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:00:19.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:00:19.237 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:19.237 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:00:19.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:20.319 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:20.313+0000 7fbdcf3a68c0 -1 Falling back to public interface 2026-02-26T03:00:20.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:20.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:20.444 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:00:20.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:00:20.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:20.445 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:00:20.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:21.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:21.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:21.646 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:00:21.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:00:21.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:21.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:00:21.855 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:22.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:22.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:22.857 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:00:22.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:00:22.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:22.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:00:23.055 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:23.608 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:23.601+0000 7fbdcf3a68c0 -1 osd.0 29 log_to_monitors true 2026-02-26T03:00:24.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:24.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:24.057 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:00:24.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:00:24.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:24.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:00:24.263 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up out weight 0 up_from 36 up_thru 0 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/3882637438,v1:127.0.0.1:6803/3882637438] [v2:127.0.0.1:6804/3882637438,v1:127.0.0.1:6805/3882637438] exists,up 93440a18-03ce-414c-9db5-dcd76945e44d 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 2 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:00:24.264 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:00:24.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:00:24.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:00:24.266 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:00:24.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:00:24.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-26T03:00:24.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:00:24.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:00:24.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:00:24.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:00:24.284 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:24.277+0000 7f7fd117c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:00:24.285 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:24.281+0000 7f7fd117c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:00:24.286 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:24.281+0000 7f7fd117c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:00:24.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:00:24.470 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:00:24.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:24.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:00:24.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:25.579 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:25.573+0000 7f7fd117c8c0 -1 Falling back to public interface 2026-02-26T03:00:25.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:25.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:25.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:00:25.674 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:00:25.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:25.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:00:25.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:26.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:26.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:26.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:00:26.877 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:00:26.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:26.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:00:27.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:28.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:28.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:28.081 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:00:28.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:00:28.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:28.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:00:28.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:00:28.645 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:00:28.637+0000 7f7fd117c8c0 -1 osd.2 29 log_to_monitors true 2026-02-26T03:00:29.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:00:29.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:00:29.284 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:00:29.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:00:29.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:00:29.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:00:29.483 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up out weight 0 up_from 38 up_thru 0 down_at 32 last_clean_interval [13,29) [v2:127.0.0.1:6810/2801558558,v1:127.0.0.1:6811/2801558558] [v2:127.0.0.1:6812/2801558558,v1:127.0.0.1:6813/2801558558] exists,up b647e66c-bee7-46bc-a1fb-3f64d96c08d0 2026-02-26T03:00:29.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:00:29.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:00:29.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:00:29.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:29.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.0 2026-02-26T03:00:29.731 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already in. 2026-02-26T03:00:29.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2026-02-26T03:00:29.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.2 2026-02-26T03:00:29.994 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already in. 2026-02-26T03:00:30.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:00:30.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: wait_for_clean 2026-02-26T03:00:30.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:00:30.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:00:30.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:00:30.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:00:30.005 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:00:30.005 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:00:30.006 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:00:30.006 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:00:30.006 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:00:30.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:00:30.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:00:30.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:00:30.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:00:30.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:00:30.066 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:00:30.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:00:30.271 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:00:30.271 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:00:30.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:00:30.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:00:30.271 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:00:30.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822660 2026-02-26T03:00:30.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822660 2026-02-26T03:00:30.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822660' 2026-02-26T03:00:30.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:00:30.348 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:00:30.407 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:00:30.409 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:00:30.409 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:00:30.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:00:30.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:00:30.410 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:00:30.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757251 2026-02-26T03:00:30.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757251 2026-02-26T03:00:30.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822660 2-163208757251' 2026-02-26T03:00:30.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:00:30.490 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-154618822660 2026-02-26T03:00:30.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:00:30.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:00:30.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-154618822660 2026-02-26T03:00:30.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:00:30.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822660 2026-02-26T03:00:30.492 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 154618822660 2026-02-26T03:00:30.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 154618822660' 2026-02-26T03:00:30.493 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:00:30.701 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822658 -lt 154618822660 2026-02-26T03:00:30.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:00:31.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:00:31.703 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:00:31.902 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822658 -lt 154618822660 2026-02-26T03:00:31.903 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:00:32.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:00:32.904 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:00:33.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-26T03:00:33.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:00:33.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-163208757251 2026-02-26T03:00:33.109 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:00:33.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:00:33.110 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-163208757251 2026-02-26T03:00:33.111 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:00:33.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757251 2026-02-26T03:00:33.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 163208757251' 2026-02-26T03:00:33.112 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 163208757251 2026-02-26T03:00:33.112 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:00:33.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-26T03:00:33.327 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:00:33.327 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:33.327 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:33.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:33.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:33.789 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:33.789 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:33.789 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:34.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:34.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:00:34.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:00:34.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:00:34.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:34.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:00:34.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:34.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:34.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:34.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:34.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:34.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:34.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:34.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:34.347 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:34.347 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:34.348 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:34.348 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:34.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:34.604 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:34.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:34.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:34.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:00:34.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:34.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:00:35.061 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:35.061 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:35.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:35.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:35.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:35.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:35.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:35.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:35.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:35.257 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:35.257 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:35.257 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:35.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:35.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:35.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:35.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:35.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:35.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:35.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:35.512 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:35.512 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:35.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:35.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:35.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:00:35.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:35.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:00:36.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:36.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:36.166 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:36.166 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:36.167 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:36.167 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:36.167 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:36.167 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:36.360 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:36.361 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:36.361 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:36.362 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:36.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:36.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:36.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:36.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:36.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:36.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:36.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:36.619 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:36.619 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:36.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:36.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:36.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:00:36.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:36.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:00:37.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:37.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:37.677 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:37.677 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:37.677 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:37.678 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:37.678 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:37.678 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:37.873 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:37.873 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:37.874 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:37.874 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:38.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:38.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:38.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:38.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:38.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:38.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:38.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:38.143 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:38.143 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:38.402 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:38.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:38.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:00:38.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:38.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:00:40.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:40.005 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:40.005 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:40.005 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:40.005 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:40.005 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:40.005 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:40.006 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:40.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:40.202 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:40.202 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:40.202 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:40.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:40.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:40.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:40.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:40.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:40.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:40.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:40.464 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:40.464 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:40.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:40.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:40.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:00:40.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:40.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:00:43.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:43.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:43.924 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:43.924 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:43.924 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:43.924 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:43.924 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:43.924 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:44.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:44.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:44.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:44.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:44.380 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:44.380 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:44.381 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:44.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:44.642 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:44.642 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:00:44.642 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:44.642 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:00:51.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:51.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:51.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:51.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:51.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:51.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:51.044 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:51.044 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:51.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:51.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:51.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:51.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:51.494 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:51.750 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:51.751 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:51.751 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:00:51.751 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:00:51.751 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:00:59.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:00:59.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:00:59.051 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:00:59.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:00:59.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:00:59.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:00:59.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:00:59.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:00:59.246 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:00:59.247 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:00:59.247 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:00:59.247 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:00:59.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:00:59.503 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:00:59.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:00:59.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:00:59.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:00:59.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "850711f3-4b71-45f1-9beb-45373748327f", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:01:00.015484+0000", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:01:00.017 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.018 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T02:59:23.741756Z", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T02:59:23.741756Z", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:01:00.019 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 41, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T02:59:23.833147+0000", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:00:30.040642+0000", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:00:28.799197+0000", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:00:29.938435+0000", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.020 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 9, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T02:59:52.973084+0000", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "off", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 1, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "26", 2026-02-26T03:01:00.021 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "0", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "selfmanaged", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [], 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {}, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "read_balance": { 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "score_type": "Fair distribution", 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "score_acting": 2.9900000095367432, 2026-02-26T03:01:00.022 INFO:tasks.workunit.client.0.trial123.stdout: "score_stable": 2.9900000095367432, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_score": 0.67000001668930054, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_acting": 2, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_stable": 2, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity_weighted": 1, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity": 1, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity_weighted": 1 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "93440a18-03ce-414c-9db5-dcd76945e44d", 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 5, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 29, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 36, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 40, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.023 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3882637438 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/3882637438", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/3882637438", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3882637438", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3882637438", 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:01:00.024 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "434d09e9-f550-42dd-bd63-e6b534a21fde", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 9, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 20, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 31, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.025 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3458897790 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3458897790", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3458897790", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3458897790", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3458897790", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "b647e66c-bee7-46bc-a1fb-3f64d96c08d0", 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:01:00.026 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 13, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 29, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 38, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 32, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.027 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2801558558 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/2801558558", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/2801558558", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2801558558", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2801558558", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:00:17.668522+0000", 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:01:00.028 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T02:59:32.808791+0000", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:00:17.943218+0000", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T02:59:41.996614+0000", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:00:18.221290+0000", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T02:59:51.294902+0000", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:01:00.029 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/3882637438,v1:127.0.0.1:6805/3882637438]", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T02:59:26.235014Z", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:01:00.030 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/3882637438,v1:127.0.0.1:6803/3882637438]", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3882637438,v1:127.0.0.1:6809/3882637438]", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3882637438,v1:127.0.0.1:6807/3882637438]", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3458897790,v1:127.0.0.1:6813/3458897790]", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:01:00.031 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T02:59:35.460568Z", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3458897790,v1:127.0.0.1:6811/3458897790]", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3458897790,v1:127.0.0.1:6817/3458897790]", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3458897790,v1:127.0.0.1:6815/3458897790]", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/2801558558,v1:127.0.0.1:6813/2801558558]", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:01:00.032 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T02:59:44.668325Z", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/2801558558,v1:127.0.0.1:6811/2801558558]", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2801558558,v1:127.0.0.1:6817/2801558558]", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2801558558,v1:127.0.0.1:6815/2801558558]", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:01:00.033 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 26, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [ 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "floor": 26 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 36 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 38 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 41, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:01:00.034 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:01:00.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:01:00.036 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:01:00.037 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:01:00.038 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stderr:report 2924774172 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 0, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T02:59:23:832938+0000", 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:01:00.039 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 1, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 72800, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:01:00.040 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 100, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 100, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:01:00.041 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 16, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 16, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "up": 2, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 2, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83916, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2112, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488884, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036617216, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2162688, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 500149, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 1, 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:01:00.042 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83916, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2112, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488884, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036617216, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2162688, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 500149, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:01:00.043 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 1, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 72800, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 100, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 100, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:01:00.044 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1228800, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 218400, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 16, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 16, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "up": 2, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 2, 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:01:00.045 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 154618822665 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 38654705678 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 163208757256 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num": 1 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 1, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.046 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 4, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:00:42.264545+0000", 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "0": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 45, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 235, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 332, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:01:00.047 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:01:00.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:01:00.048 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: expr 2 + 2 2026-02-26T03:01:00.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: objname=existing_4 2026-02-26T03:01:00.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: echo 'writing non-divergent object existing_4' 2026-02-26T03:01:00.048 INFO:tasks.workunit.client.0.trial123.stdout:writing non-divergent object existing_4 2026-02-26T03:01:00.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:155: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:00.217 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:00.218 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 29'100 41:132 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:00.218 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:00.218 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:00.218 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:00.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:156: TEST_divergent: rados -p test put existing_4 /etc/resolv.conf 2026-02-26T03:01:00.255 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery 2026-02-26T03:01:00.255 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: echo 'delay recovery' 2026-02-26T03:01:00.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:160: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:00.445 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:00.446 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 29'100 41:132 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:00.446 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:00.446 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:00.446 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:00.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2026-02-26T03:01:00.454 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.0 2026-02-26T03:01:00.454 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:01:00.454 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:01:00.455 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:01:00.455 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:00.455 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:00.455 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:01:00.455 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2026-02-26T03:01:00.455 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok set_recovery_delay 100000 2026-02-26T03:01:00.508 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:01:00.513 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.2 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:00.514 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:01:00.515 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2026-02-26T03:01:00.515 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok set_recovery_delay 100000 2026-02-26T03:01:00.567 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:01:00.573 INFO:tasks.workunit.client.0.trial123.stdout:revive divergent 1 2026-02-26T03:01:00.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:167: TEST_divergent: echo 'revive divergent 1' 2026-02-26T03:01:00.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:168: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:00.774 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:00.774 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 29'100 41:132 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:00.774 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:00.774 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:00.774 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:00.782 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:169: TEST_divergent: ceph osd set noup 2026-02-26T03:01:01.096 INFO:tasks.workunit.client.0.trial123.stderr:noup is set 2026-02-26T03:01:01.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:170: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-26T03:01:01.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:01:01.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:01:01.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:01:01.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:01:01.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:01:01.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:01:01.113 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:01:01.113 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:01:01.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:01:01.114 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:01:01.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:01:01.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:01:01.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:01:01.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:01:01.135 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:01.128+0000 7f93628d68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:01.136 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:01.128+0000 7f93628d68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:01.137 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:01.132+0000 7f93628d68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:01.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-26T03:01:01.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:171: TEST_divergent: sleep 5 2026-02-26T03:01:02.367 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:02.360+0000 7f93628d68c0 -1 Falling back to public interface 2026-02-26T03:01:05.338 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:05.332+0000 7f93628d68c0 -1 osd.1 29 log_to_monitors true 2026-02-26T03:01:06.312 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery divergent 2026-02-26T03:01:06.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: echo 'delay recovery divergent' 2026-02-26T03:01:06.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:174: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:06.498 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:06.499 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 41'101 43:137 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:06.499 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:06.499 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:06.499 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:06.508 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: get_asok_path osd.1 2026-02-26T03:01:06.508 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:01:06.508 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:01:06.508 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:01:06.508 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:06.508 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:06.509 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:01:06.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: CEPH_ARGS= 2026-02-26T03:01:06.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok set_recovery_delay 100000 2026-02-26T03:01:06.561 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:01:06.567 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:177: TEST_divergent: ceph osd unset noup 2026-02-26T03:01:06.880 INFO:tasks.workunit.client.0.trial123.stderr:noup is unset 2026-02-26T03:01:06.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:179: TEST_divergent: wait_for_osd up 0 2026-02-26T03:01:06.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:06.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:01:06.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:06.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:06.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:06.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:06.894 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:06.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:06.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:01:07.092 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 36 up_thru 40 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/3882637438,v1:127.0.0.1:6803/3882637438] [v2:127.0.0.1:6804/3882637438,v1:127.0.0.1:6805/3882637438] exists,up 93440a18-03ce-414c-9db5-dcd76945e44d 2026-02-26T03:01:07.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:07.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:180: TEST_divergent: wait_for_osd up 1 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:07.094 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:07.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:07.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:07.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:07.288 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 31 last_clean_interval [9,29) [v2:127.0.0.1:6818/3842329129,v1:127.0.0.1:6819/3842329129] [v2:127.0.0.1:6820/3842329129,v1:127.0.0.1:6821/3842329129] exists,up 434d09e9-f550-42dd-bd63-e6b534a21fde 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:181: TEST_divergent: wait_for_osd up 2 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:07.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:07.487 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 38 up_thru 0 down_at 32 last_clean_interval [13,29) [v2:127.0.0.1:6810/2801558558,v1:127.0.0.1:6811/2801558558] [v2:127.0.0.1:6812/2801558558,v1:127.0.0.1:6813/2801558558] exists,up b647e66c-bee7-46bc-a1fb-3f64d96c08d0 2026-02-26T03:01:07.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:07.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:07.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:07.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:183: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:07.671 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:07.671 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 41'101 43:137 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:07.672 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:07.672 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:07.672 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:07.680 INFO:tasks.workunit.client.0.trial123.stdout:wait for peering 2026-02-26T03:01:07.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:184: TEST_divergent: echo 'wait for peering' 2026-02-26T03:01:07.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:185: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:07.879 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:07.879 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 41'101 43:137 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:07.880 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:07.880 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:07.880 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:07.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:186: TEST_divergent: rados -p test put foo /etc/fstab 2026-02-26T03:01:07.918 INFO:tasks.workunit.client.0.trial123.stdout:killing divergent 1 2026-02-26T03:01:07.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:188: TEST_divergent: echo 'killing divergent 1' 2026-02-26T03:01:07.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:189: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:08.109 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:08.109 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 41'101 43:137 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:08.109 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:08.109 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:08.110 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:08.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:190: TEST_divergent: kill_daemons td/divergent-priors KILL osd.1 2026-02-26T03:01:08.120 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:01:08.120 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:01:08.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:01:08.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:01:08.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:01:08.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:01:08.229 INFO:tasks.workunit.client.0.trial123.stdout:reviving divergent 1 2026-02-26T03:01:08.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:192: TEST_divergent: echo 'reviving divergent 1' 2026-02-26T03:01:08.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:193: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:08.417 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:08.417 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:00:30.043870+0000 41'101 43:137 [0,2] 0 [0,2] 0 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:40:33.684067+0000 0 0 2026-02-26T03:01:08.417 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:08.417 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:08.418 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:194: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:01:08.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:08.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:01:08.430 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:01:08.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:01:08.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:01:08.431 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:01:08.432 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:01:08.432 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:01:08.432 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:01:08.433 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:01:08.453 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:08.444+0000 7fd7793d38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:08.454 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:08.448+0000 7fd7793d38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:08.455 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:08.448+0000 7fd7793d38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:08.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:01:08.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:08.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:01:08.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:08.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:08.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:08.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:08.630 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:08.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:08.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:08.829 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 31 last_clean_interval [9,29) [v2:127.0.0.1:6818/3842329129,v1:127.0.0.1:6819/3842329129] [v2:127.0.0.1:6820/3842329129,v1:127.0.0.1:6821/3842329129] exists,up 434d09e9-f550-42dd-bd63-e6b534a21fde 2026-02-26T03:01:08.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:08.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:08.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:08.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:196: TEST_divergent: sleep 20 2026-02-26T03:01:09.959 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:09.952+0000 7fd7793d38c0 -1 Falling back to public interface 2026-02-26T03:01:12.993 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:12.988+0000 7fd7793d38c0 -1 osd.1 47 log_to_monitors true 2026-02-26T03:01:28.830 INFO:tasks.workunit.client.0.trial123.stdout:allowing recovery 2026-02-26T03:01:28.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:198: TEST_divergent: echo 'allowing recovery' 2026-02-26T03:01:28.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:29.016 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:29.017 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 0 0 0 73767 0 0 18 84 18 active+clean 2026-02-26T03:01:13.562879+0000 47'102 51:161 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T07:07:50.132590+0000 0 0 2026-02-26T03:01:29.017 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:29.017 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:29.017 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:29.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-26T03:01:29.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-26T03:01:29.111 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:01:29.118 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-26T03:01:29.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-26T03:01:29.198 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:01:29.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-26T03:01:29.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-26T03:01:29.282 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:01:29.289 INFO:tasks.workunit.client.0.trial123.stdout:reading divergent objects 2026-02-26T03:01:29.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: echo 'reading divergent objects' 2026-02-26T03:01:29.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:207: TEST_divergent: ceph pg dump pgs 2026-02-26T03:01:29.486 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:01:29.486 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 0 0 0 73767 0 0 18 84 18 active+clean 2026-02-26T03:01:13.562879+0000 47'102 51:161 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T02:59:53.010199+0000 0'0 2026-02-26T02:59:53.010199+0000 0 0 periodic scrub scheduled @ 2026-02-27T07:07:50.132590+0000 0 0 2026-02-26T03:01:29.486 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:01:29.486 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:01:29.487 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:01:29.495 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: expr 2 + 2 2026-02-26T03:01:29.496 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: seq 1 4 2026-02-26T03:01:29.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:01:29.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_1 td/divergent-priors/existing 2026-02-26T03:01:29.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:01:29.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_2 td/divergent-priors/existing 2026-02-26T03:01:29.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:01:29.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_3 td/divergent-priors/existing 2026-02-26T03:01:29.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:01:29.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_4 td/divergent-priors/existing 2026-02-26T03:01:29.600 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: rm -f td/divergent-priors/existing 2026-02-26T03:01:29.601 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-26T03:01:29.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4552.0:1 2026-02-26T03:00:06.793297+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 20 _merge_object_divergent_entries: keeping 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4552.0:1 2026-02-26T03:00:06.793297+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 29'100 first_divergent_update: 29'101 last_divergent_update: 29'101 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 29'101 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4552.0:1 2026-02-26T03:00:06.793297+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:01:07.204+0000 7f934485c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-26T03:01:29.608 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-26T03:01:29.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: grep -q '_merge_object_divergent_entries.*cannot roll back, removing and adding to missing' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:01:29.612 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:221: TEST_divergent: echo success 2026-02-26T03:01:29.613 INFO:tasks.workunit.client.0.trial123.stdout:success 2026-02-26T03:01:29.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:223: TEST_divergent: delete_pool test 2026-02-26T03:01:29.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-26T03:01:29.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-26T03:01:29.867 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' does not exist 2026-02-26T03:01:29.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:224: TEST_divergent: kill_daemons td/divergent-priors 2026-02-26T03:01:29.878 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:01:29.878 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:01:29.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:01:29.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:01:29.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:01:31.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:01:31.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:01:31.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:01:31.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:01:31.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:01:31.196 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:01:31.197 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:01:31.197 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:01:31.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:01:31.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:01:31.199 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:01:31.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:01:31.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:01:31.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:01:31.201 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:01:31.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:01:31.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:01:31.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:01:31.217 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:01:31.217 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.217 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.217 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:01:31.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:01:31.220 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:01:31.220 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:01:31.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:01:31.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:01:31.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:01:31.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:01:31.222 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:01:31.223 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:01:31.223 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:01:31.225 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:01:31.225 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:01:31.225 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:01:31.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:01:31.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:01:31.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:01:31.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:01:31.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:01:31.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:01:31.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:01:31.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:01:31.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.230 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:01:31.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:01:31.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:01:31.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-26T03:01:31.233 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:01:31.233 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.233 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.233 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.18886 2026-02-26T03:01:31.235 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:01:31.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-26T03:01:31.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:01:31.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-26T03:01:31.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_2 td/divergent-priors 2026-02-26T03:01:31.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:421: TEST_divergent_2: local dir=td/divergent-priors 2026-02-26T03:01:31.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: local dummyfile=/etc/fstab 2026-02-26T03:01:31.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:425: TEST_divergent_2: local dummyfile2=/etc/resolv.conf 2026-02-26T03:01:31.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:427: TEST_divergent_2: local num_osds=3 2026-02-26T03:01:31.236 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: expr 3 - 1 2026-02-26T03:01:31.237 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: seq 0 2 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: local 'osds=0 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: run_mon td/divergent-priors a 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:01:31.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-26T03:01:31.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:31.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:01:31.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:01:31.304 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:01:31.304 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:01:31.304 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:01:31.304 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.305 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.306 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T03:01:31.306 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:01:31.306 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get fsid 2026-02-26T03:01:31.362 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:01:31.362 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:01:31.362 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:01:31.362 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:01:31.362 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:01:31.363 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:01:31.363 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:01:31.363 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:01:31.363 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:01:31.363 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.363 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.364 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T03:01:31.364 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:01:31.364 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get mon_host 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:430: TEST_divergent_2: run_mgr td/divergent-priors x 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-26T03:01:31.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:01:31.537 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:01:31.537 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:31.538 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:31.538 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:31.538 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.538 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.538 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:31.539 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:01:31.539 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 0 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:01:31.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:31.561 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:01:31.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9494153f-5bc6-4a02-8f18-4660dcead5ca 2026-02-26T03:01:31.563 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 9494153f-5bc6-4a02-8f18-4660dcead5ca 2026-02-26T03:01:31.564 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 9494153f-5bc6-4a02-8f18-4660dcead5ca' 2026-02-26T03:01:31.564 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:01:31.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCLt59p60dGIhAAG3d+JXtKfiuzpoBjGiViAQ== 2026-02-26T03:01:31.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCLt59p60dGIhAAG3d+JXtKfiuzpoBjGiViAQ=="}' 2026-02-26T03:01:31.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9494153f-5bc6-4a02-8f18-4660dcead5ca -i td/divergent-priors/0/new.json 2026-02-26T03:01:31.687 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:31.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-26T03:01:31.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCLt59p60dGIhAAG3d+JXtKfiuzpoBjGiViAQ== --osd-uuid 9494153f-5bc6-4a02-8f18-4660dcead5ca 2026-02-26T03:01:31.715 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:31.709+0000 7fc7e54f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:31.716 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:31.713+0000 7fc7e54f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:31.717 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:31.713+0000 7fc7e54f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:31.718 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:31.713+0000 7fc7e54f88c0 -1 bdev(0x55f0d5601c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:01:31.718 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:31.713+0000 7fc7e54f88c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-26T03:01:34.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-26T03:01:34.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:01:34.337 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:01:34.337 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:01:34.337 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:01:34.619 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:01:34.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:01:34.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:01:34.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:01:34.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:01:34.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:01:34.639 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:34.633+0000 7f80cc0308c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:34.640 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:34.637+0000 7f80cc0308c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:34.642 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:34.637+0000 7f80cc0308c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:34.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:34.822 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:34.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:34.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:01:35.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:36.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:36.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:36.024 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:01:36.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:01:36.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:36.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:01:36.132 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:36.129+0000 7f80cc0308c0 -1 Falling back to public interface 2026-02-26T03:01:36.227 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:37.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:37.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:37.228 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:01:37.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:01:37.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:37.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:01:37.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:38.435 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:38.435 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:38.436 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:01:38.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:01:38.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:38.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:01:38.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:39.123 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:39.121+0000 7f80cc0308c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:01:39.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:39.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:39.640 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:01:39.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:01:39.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:39.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:01:39.842 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1255639116,v1:127.0.0.1:6803/1255639116] [v2:127.0.0.1:6804/1255639116,v1:127.0.0.1:6805/1255639116] exists,up 9494153f-5bc6-4a02-8f18-4660dcead5ca 2026-02-26T03:01:39.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:39.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:39.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:39.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-26T03:01:39.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 1 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:39.844 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:01:39.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:01:39.846 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:01:39.847 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:01:39.847 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:01:39.847 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 9554fde6-99f6-48e4-b715-1384b74549a4' 2026-02-26T03:01:39.847 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:01:39.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCTt59pDU9dMxAA6+NdQygr14WDFjn0yYCiRg== 2026-02-26T03:01:39.862 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCTt59pDU9dMxAA6+NdQygr14WDFjn0yYCiRg=="}' 2026-02-26T03:01:39.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9554fde6-99f6-48e4-b715-1384b74549a4 -i td/divergent-priors/1/new.json 2026-02-26T03:01:40.072 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:01:40.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-26T03:01:40.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCTt59pDU9dMxAA6+NdQygr14WDFjn0yYCiRg== --osd-uuid 9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:01:40.102 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:40.097+0000 7fcd5e1298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:40.103 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:40.101+0000 7fcd5e1298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:40.104 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:40.101+0000 7fcd5e1298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:40.105 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:40.101+0000 7fcd5e1298c0 -1 bdev(0x558423425c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-26T03:01:40.105 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:40.101+0000 7fcd5e1298c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-26T03:01:42.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-26T03:01:42.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:01:42.377 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T03:01:42.377 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T03:01:42.377 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:01:42.653 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:01:42.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T03:01:42.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:01:42.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:01:42.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:01:42.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:01:42.681 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:42.677+0000 7f4942a278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:42.683 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:42.681+0000 7f4942a278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:42.684 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:42.681+0000 7f4942a278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:42.853 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T03:01:42.853 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:42.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:01:42.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:42.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:42.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:42.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:42.855 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:42.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:42.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:43.056 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:44.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:44.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:44.059 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:01:44.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:01:44.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:44.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:44.177 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:44.173+0000 7f4942a278c0 -1 Falling back to public interface 2026-02-26T03:01:44.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:45.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:45.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:45.263 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:01:45.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:01:45.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:45.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:45.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:46.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:46.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:46.466 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:01:46.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:01:46.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:46.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:46.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:47.403 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:47.401+0000 7f4942a278c0 -1 osd.1 0 log_to_monitors true 2026-02-26T03:01:47.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:47.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:47.672 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:01:47.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:01:47.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:47.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:47.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:48.877 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:48.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:48.878 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:01:48.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:01:48.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:48.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:01:49.079 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3656840459,v1:127.0.0.1:6811/3656840459] [v2:127.0.0.1:6812/3656840459,v1:127.0.0.1:6813/3656840459] exists,up 9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:01:49.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:49.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:49.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:49.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-26T03:01:49.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 2 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:01:49.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:01:49.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:01:49.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:01:49.086 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:01:49.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a75b1a3a-0132-4f1c-babe-e4d4ec680aec 2026-02-26T03:01:49.086 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 a75b1a3a-0132-4f1c-babe-e4d4ec680aec 2026-02-26T03:01:49.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a75b1a3a-0132-4f1c-babe-e4d4ec680aec' 2026-02-26T03:01:49.087 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:01:49.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCdt59pm8/eBRAA6OWxOVVtxyK5DM2KPQ9usw== 2026-02-26T03:01:49.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCdt59pm8/eBRAA6OWxOVVtxyK5DM2KPQ9usw=="}' 2026-02-26T03:01:49.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a75b1a3a-0132-4f1c-babe-e4d4ec680aec -i td/divergent-priors/2/new.json 2026-02-26T03:01:49.319 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:01:49.328 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-26T03:01:49.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCdt59pm8/eBRAA6OWxOVVtxyK5DM2KPQ9usw== --osd-uuid a75b1a3a-0132-4f1c-babe-e4d4ec680aec 2026-02-26T03:01:49.348 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:49.345+0000 7f8df0a438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:49.349 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:49.345+0000 7f8df0a438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:49.350 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:49.349+0000 7f8df0a438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:49.351 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:49.349+0000 7f8df0a438c0 -1 bdev(0x55fa8966bc00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-26T03:01:49.351 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:49.349+0000 7f8df0a438c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-26T03:01:51.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-26T03:01:51.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:01:51.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T03:01:51.640 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T03:01:51.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:01:51.918 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T03:01:51.919 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:01:51.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:01:51.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:01:51.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:01:51.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:01:51.947 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:51.945+0000 7efeb2f3a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:51.948 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:51.945+0000 7efeb2f3a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:51.949 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:51.945+0000 7efeb2f3a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:01:52.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T03:01:52.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:01:52.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:01:52.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:01:52.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:01:52.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:52.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:01:52.120 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:01:52.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:52.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:52.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:53.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:53.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:53.327 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:01:53.327 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:01:53.327 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:53.327 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:53.456 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:53.453+0000 7efeb2f3a8c0 -1 Falling back to public interface 2026-02-26T03:01:53.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:54.532 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:54.532 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:54.533 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:01:54.533 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:01:54.533 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:54.533 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:54.733 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:55.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:55.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:55.735 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:01:55.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:01:55.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:55.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:55.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:56.695 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:01:56.693+0000 7efeb2f3a8c0 -1 osd.2 0 log_to_monitors true 2026-02-26T03:01:56.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:56.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:56.939 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:01:56.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:01:56.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:56.940 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:57.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:01:58.146 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:01:58.146 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:01:58.147 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:01:58.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:01:58.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:01:58.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:01:58.347 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3076437224,v1:127.0.0.1:6819/3076437224] [v2:127.0.0.1:6820/3076437224,v1:127.0.0.1:6821/3076437224] exists,up a75b1a3a-0132-4f1c-babe-e4d4ec680aec 2026-02-26T03:01:58.347 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:01:58.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:01:58.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:01:58.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:436: TEST_divergent_2: ceph osd set noout 2026-02-26T03:01:58.650 INFO:tasks.workunit.client.0.trial123.stderr:noout is set 2026-02-26T03:01:58.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:437: TEST_divergent_2: ceph osd set noin 2026-02-26T03:01:58.955 INFO:tasks.workunit.client.0.trial123.stderr:noin is set 2026-02-26T03:01:58.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:438: TEST_divergent_2: ceph osd set nodown 2026-02-26T03:01:59.258 INFO:tasks.workunit.client.0.trial123.stderr:nodown is set 2026-02-26T03:01:59.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:439: TEST_divergent_2: create_pool test 1 1 2026-02-26T03:01:59.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-26T03:01:59.512 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' already exists 2026-02-26T03:01:59.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T03:02:00.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:440: TEST_divergent_2: ceph osd pool set test size 3 2026-02-26T03:02:00.816 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 size to 3 2026-02-26T03:02:00.827 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:441: TEST_divergent_2: ceph osd pool set test min_size 2 2026-02-26T03:02:01.119 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 min_size to 2 2026-02-26T03:02:01.132 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:443: TEST_divergent_2: flush_pg_stats 2026-02-26T03:02:01.132 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:02:01.132 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:02:01.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:02:01.333 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:02:01.333 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:02:01.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:02:01.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:01.334 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:02:01.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-26T03:02:01.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-26T03:02:01.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-26T03:02:01.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:01.412 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:02:01.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-26T03:02:01.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-26T03:02:01.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-26T03:02:01.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:01.492 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:02:01.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-26T03:02:01.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-26T03:02:01.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-60129542147' 2026-02-26T03:02:01.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:01.575 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-26T03:02:01.575 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:01.577 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:02:01.577 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-26T03:02:01.578 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:01.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-26T03:02:01.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-26T03:02:01.579 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836487 2026-02-26T03:02:01.579 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:02:01.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836487 2026-02-26T03:02:01.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:02:02.776 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:02:02.777 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:02:02.980 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-26T03:02:02.981 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:02.981 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-26T03:02:02.981 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:02.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:02:02.983 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-26T03:02:02.983 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:02.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-26T03:02:02.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-26T03:02:02.985 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 38654705669 2026-02-26T03:02:02.986 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:02:03.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-26T03:02:03.183 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:03.183 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-26T03:02:03.183 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:03.184 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:02:03.185 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-26T03:02:03.185 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:03.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-26T03:02:03.186 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542147 2026-02-26T03:02:03.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-26T03:02:03.187 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:02:03.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-26T03:02:03.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:444: TEST_divergent_2: wait_for_clean 2026-02-26T03:02:03.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:02:03.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:02:03.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:02:03.393 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:02:03.394 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:02:03.394 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:02:03.394 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:02:03.394 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:02:03.394 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:02:03.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:02:03.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:02:03.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:02:03.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:02:03.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:02:03.490 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:02:03.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:02:03.695 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:02:03.696 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:02:03.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:02:03.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:03.696 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:02:03.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-26T03:02:03.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-26T03:02:03.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-26T03:02:03.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:03.776 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:02:03.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-26T03:02:03.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-26T03:02:03.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-26T03:02:03.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:03.857 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:02:03.934 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-26T03:02:03.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-26T03:02:03.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-60129542150' 2026-02-26T03:02:03.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:03.935 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-26T03:02:03.935 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:03.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:02:03.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-26T03:02:03.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:03.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-26T03:02:03.938 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836489 2026-02-26T03:02:03.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-26T03:02:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:02:04.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-26T03:02:04.139 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:04.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-26T03:02:04.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:04.140 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:02:04.141 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-26T03:02:04.141 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:04.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-26T03:02:04.143 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 38654705672 2026-02-26T03:02:04.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-26T03:02:04.143 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:02:04.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705672 2026-02-26T03:02:04.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:02:05.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:02:05.343 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:02:05.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705672 2026-02-26T03:02:05.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:02:06.548 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:02:06.548 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:02:06.750 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-26T03:02:06.750 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:06.751 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-26T03:02:06.751 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:06.752 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:02:06.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-26T03:02:06.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:06.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-26T03:02:06.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-26T03:02:06.755 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542150 2026-02-26T03:02:06.755 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:02:06.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-26T03:02:06.963 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:02:06.963 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:06.963 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:07.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:02:07.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:07.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:07.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:07.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:07.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:07.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:07.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:07.427 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:02:07.427 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:07.428 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:07.428 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:07.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:02:07.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:02:07.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:02:07.695 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: ceph pg dump pgs --format=json 2026-02-26T03:02:07.695 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: jq '.pg_stats[0].up_primary' 2026-02-26T03:02:07.886 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:02:07.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: local divergent=1 2026-02-26T03:02:07.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: echo 'primary and soon to be divergent is 1' 2026-02-26T03:02:07.896 INFO:tasks.workunit.client.0.trial123.stdout:primary and soon to be divergent is 1 2026-02-26T03:02:07.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:449: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:02:08.084 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:02:08.084 INFO:tasks.workunit.client.0.trial123.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-26T03:01:59.625423+0000 0'0 27:23 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:14:55.773702+0000 0 0 2026-02-26T03:02:08.085 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:02:08.085 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:02:08.085 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:450: TEST_divergent_2: local non_divergent= 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 0 = 1 ']' 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0' 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 1 = 1 ']' 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: continue 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 2 = 1 ']' 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0 2' 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:459: TEST_divergent_2: echo 'writing initial objects' 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stdout:writing initial objects 2026-02-26T03:02:08.093 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: seq 1 100 2026-02-26T03:02:08.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_1 /etc/fstab 2026-02-26T03:02:08.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_2 /etc/fstab 2026-02-26T03:02:08.145 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.145 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_3 /etc/fstab 2026-02-26T03:02:08.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_4 /etc/fstab 2026-02-26T03:02:08.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_5 /etc/fstab 2026-02-26T03:02:08.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_6 /etc/fstab 2026-02-26T03:02:08.257 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.257 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_7 /etc/fstab 2026-02-26T03:02:08.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_8 /etc/fstab 2026-02-26T03:02:08.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_9 /etc/fstab 2026-02-26T03:02:08.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_10 /etc/fstab 2026-02-26T03:02:08.361 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.361 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_11 /etc/fstab 2026-02-26T03:02:08.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_12 /etc/fstab 2026-02-26T03:02:08.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_13 /etc/fstab 2026-02-26T03:02:08.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_14 /etc/fstab 2026-02-26T03:02:08.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_15 /etc/fstab 2026-02-26T03:02:08.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_16 /etc/fstab 2026-02-26T03:02:08.527 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.527 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_17 /etc/fstab 2026-02-26T03:02:08.556 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.556 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_18 /etc/fstab 2026-02-26T03:02:08.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_19 /etc/fstab 2026-02-26T03:02:08.612 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.612 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_20 /etc/fstab 2026-02-26T03:02:08.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_21 /etc/fstab 2026-02-26T03:02:08.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_22 /etc/fstab 2026-02-26T03:02:08.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_23 /etc/fstab 2026-02-26T03:02:08.714 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.715 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_24 /etc/fstab 2026-02-26T03:02:08.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_25 /etc/fstab 2026-02-26T03:02:08.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_26 /etc/fstab 2026-02-26T03:02:08.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_27 /etc/fstab 2026-02-26T03:02:08.824 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.824 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_28 /etc/fstab 2026-02-26T03:02:08.852 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.852 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_29 /etc/fstab 2026-02-26T03:02:08.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_30 /etc/fstab 2026-02-26T03:02:08.908 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.909 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_31 /etc/fstab 2026-02-26T03:02:08.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_32 /etc/fstab 2026-02-26T03:02:08.963 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.964 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_33 /etc/fstab 2026-02-26T03:02:08.992 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:08.992 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_34 /etc/fstab 2026-02-26T03:02:09.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_35 /etc/fstab 2026-02-26T03:02:09.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_36 /etc/fstab 2026-02-26T03:02:09.075 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.075 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_37 /etc/fstab 2026-02-26T03:02:09.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_38 /etc/fstab 2026-02-26T03:02:09.131 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.131 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_39 /etc/fstab 2026-02-26T03:02:09.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_40 /etc/fstab 2026-02-26T03:02:09.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_41 /etc/fstab 2026-02-26T03:02:09.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_42 /etc/fstab 2026-02-26T03:02:09.234 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.234 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_43 /etc/fstab 2026-02-26T03:02:09.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_44 /etc/fstab 2026-02-26T03:02:09.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_45 /etc/fstab 2026-02-26T03:02:09.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_46 /etc/fstab 2026-02-26T03:02:09.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_47 /etc/fstab 2026-02-26T03:02:09.358 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.358 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_48 /etc/fstab 2026-02-26T03:02:09.384 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.384 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_49 /etc/fstab 2026-02-26T03:02:09.412 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.412 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_50 /etc/fstab 2026-02-26T03:02:09.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_51 /etc/fstab 2026-02-26T03:02:09.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_52 /etc/fstab 2026-02-26T03:02:09.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_53 /etc/fstab 2026-02-26T03:02:09.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_54 /etc/fstab 2026-02-26T03:02:09.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_55 /etc/fstab 2026-02-26T03:02:09.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_56 /etc/fstab 2026-02-26T03:02:09.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_57 /etc/fstab 2026-02-26T03:02:09.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_58 /etc/fstab 2026-02-26T03:02:09.644 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_59 /etc/fstab 2026-02-26T03:02:09.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_60 /etc/fstab 2026-02-26T03:02:09.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_61 /etc/fstab 2026-02-26T03:02:09.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_62 /etc/fstab 2026-02-26T03:02:09.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_63 /etc/fstab 2026-02-26T03:02:09.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_64 /etc/fstab 2026-02-26T03:02:09.811 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_65 /etc/fstab 2026-02-26T03:02:09.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_66 /etc/fstab 2026-02-26T03:02:09.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_67 /etc/fstab 2026-02-26T03:02:09.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_68 /etc/fstab 2026-02-26T03:02:09.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_69 /etc/fstab 2026-02-26T03:02:09.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_70 /etc/fstab 2026-02-26T03:02:09.979 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:09.979 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_71 /etc/fstab 2026-02-26T03:02:10.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_72 /etc/fstab 2026-02-26T03:02:10.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_73 /etc/fstab 2026-02-26T03:02:10.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.061 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_74 /etc/fstab 2026-02-26T03:02:10.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_75 /etc/fstab 2026-02-26T03:02:10.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_76 /etc/fstab 2026-02-26T03:02:10.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_77 /etc/fstab 2026-02-26T03:02:10.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_78 /etc/fstab 2026-02-26T03:02:10.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_79 /etc/fstab 2026-02-26T03:02:10.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_80 /etc/fstab 2026-02-26T03:02:10.255 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_81 /etc/fstab 2026-02-26T03:02:10.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_82 /etc/fstab 2026-02-26T03:02:10.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_83 /etc/fstab 2026-02-26T03:02:10.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_84 /etc/fstab 2026-02-26T03:02:10.367 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_85 /etc/fstab 2026-02-26T03:02:10.395 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.396 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_86 /etc/fstab 2026-02-26T03:02:10.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_87 /etc/fstab 2026-02-26T03:02:10.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_88 /etc/fstab 2026-02-26T03:02:10.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_89 /etc/fstab 2026-02-26T03:02:10.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_90 /etc/fstab 2026-02-26T03:02:10.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_91 /etc/fstab 2026-02-26T03:02:10.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_92 /etc/fstab 2026-02-26T03:02:10.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.592 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_93 /etc/fstab 2026-02-26T03:02:10.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_94 /etc/fstab 2026-02-26T03:02:10.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_95 /etc/fstab 2026-02-26T03:02:10.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_96 /etc/fstab 2026-02-26T03:02:10.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_97 /etc/fstab 2026-02-26T03:02:10.732 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.732 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_98 /etc/fstab 2026-02-26T03:02:10.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_99 /etc/fstab 2026-02-26T03:02:10.787 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-26T03:02:10.788 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_100 /etc/fstab 2026-02-26T03:02:10.815 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:02:10.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: wait_for_clean 2026-02-26T03:02:10.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:02:10.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:02:10.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:02:10.816 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:02:10.817 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:02:10.817 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:02:10.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:02:10.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:02:10.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:02:10.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:02:10.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:02:10.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:02:10.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:02:10.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:02:10.889 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:02:11.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:02:11.095 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:02:11.095 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:02:11.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:02:11.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:11.096 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:02:11.179 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-26T03:02:11.179 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-26T03:02:11.179 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-26T03:02:11.179 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:11.179 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:02:11.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-26T03:02:11.257 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-26T03:02:11.257 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675' 2026-02-26T03:02:11.257 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:11.257 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:02:11.337 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-26T03:02:11.338 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-26T03:02:11.338 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675 2-60129542153' 2026-02-26T03:02:11.338 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:11.338 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-26T03:02:11.338 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:11.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:02:11.340 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-26T03:02:11.340 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:11.341 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-26T03:02:11.341 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836493 2026-02-26T03:02:11.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-26T03:02:11.342 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:02:11.548 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836493 2026-02-26T03:02:11.548 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:02:12.549 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:02:12.550 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:02:12.752 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-26T03:02:12.752 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:12.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-26T03:02:12.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:12.754 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:02:12.755 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-26T03:02:12.755 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:12.757 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-26T03:02:12.757 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 38654705675 2026-02-26T03:02:12.757 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-26T03:02:12.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:02:12.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-26T03:02:12.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:12.968 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-26T03:02:12.968 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:12.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:02:12.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-26T03:02:12.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:12.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-26T03:02:12.971 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542153 2026-02-26T03:02:12.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-26T03:02:12.972 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:02:13.177 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-26T03:02:13.178 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:02:13.178 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:13.178 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:13.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:13.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:13.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:02:13.646 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:13.646 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:13.646 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:13.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:02:13.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:02:13.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:02:13.913 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: get_pg test existing_1 2026-02-26T03:02:13.913 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-26T03:02:13.914 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-26T03:02:13.914 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-26T03:02:13.914 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-26T03:02:14.123 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: local pgid=1.0 2026-02-26T03:02:14.123 INFO:tasks.workunit.client.0.trial123.stdout:blackholing osds 0 2 2026-02-26T03:02:14.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:471: TEST_divergent_2: echo 'blackholing osds 0 2' 2026-02-26T03:02:14.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:472: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:02:14.311 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:02:14.312 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-26T03:01:59.625423+0000 27'100 27:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:14:55.773702+0000 0 0 2026-02-26T03:02:14.312 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:02:14.312 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:02:14.312 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:02:14.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:14.320 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.0 2026-02-26T03:02:14.320 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:02:14.320 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:02:14.321 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:02:14.321 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:02:14.321 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:02:14.321 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:02:14.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2026-02-26T03:02:14.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-26T03:02:14.372 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:02:14.372 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:02:14.372 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:02:14.377 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:14.378 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.2 2026-02-26T03:02:14.378 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:02:14.378 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:02:14.378 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:02:14.378 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:02:14.378 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:02:14.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:02:14.379 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2026-02-26T03:02:14.379 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-26T03:02:14.430 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:02:14.430 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:02:14.430 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:02:14.435 INFO:tasks.workunit.client.0.trial123.stdout:create new divergent objects 2026-02-26T03:02:14.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:479: TEST_divergent_2: echo 'create new divergent objects' 2026-02-26T03:02:14.436 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: seq 1 2 2026-02-26T03:02:14.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:02:14.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:02:14.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_1 2026-02-26T03:02:14.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:485: TEST_divergent_2: echo 'writing divergent objects' 2026-02-26T03:02:14.437 INFO:tasks.workunit.client.0.trial123.stdout:writing divergent objects 2026-02-26T03:02:14.438 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_2 2026-02-26T03:02:14.438 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: seq 1 2 2026-02-26T03:02:14.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-26T03:02:14.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-26T03:02:14.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_1 /etc/resolv.conf 2026-02-26T03:02:14.439 INFO:tasks.workunit.client.0.trial123.stdout:remove divergent objects 2026-02-26T03:02:14.440 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:491: TEST_divergent_2: echo 'remove divergent objects' 2026-02-26T03:02:14.440 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_2 /etc/resolv.conf 2026-02-26T03:02:14.440 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: seq 1 2 2026-02-26T03:02:14.440 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-26T03:02:14.441 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 1 + 2 2026-02-26T03:02:14.442 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=3 2026-02-26T03:02:14.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-26T03:02:14.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_3 2026-02-26T03:02:14.443 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 2 + 2 2026-02-26T03:02:14.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=4 2026-02-26T03:02:14.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:497: TEST_divergent_2: sleep 10 2026-02-26T03:02:14.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_4 2026-02-26T03:02:24.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:498: TEST_divergent_2: killall -9 rados 2026-02-26T03:02:24.492 INFO:tasks.workunit.client.0.trial123.stderr:rados: no process found 2026-02-26T03:02:24.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:501: TEST_divergent_2: echo 'killing all the osds' 2026-02-26T03:02:24.493 INFO:tasks.workunit.client.0.trial123.stdout:killing all the osds 2026-02-26T03:02:24.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:502: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:02:24.696 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:02:24.697 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 active+clean 2026-02-26T03:01:59.625423+0000 27'106 27:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:14:55.773702+0000 0 0 2026-02-26T03:02:24.697 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:02:24.697 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:02:24.697 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:02:24.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:503: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd 2026-02-26T03:02:24.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:02:24.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:02:24.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:02:24.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:02:24.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:02:24.815 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:02:24.815 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-26T03:02:24.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.0 2026-02-26T03:02:25.068 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:02:25.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-26T03:02:25.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.1 2026-02-26T03:02:25.330 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:02:25.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-26T03:02:25.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.2 2026-02-26T03:02:25.597 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:02:25.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:25.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.0 2026-02-26T03:02:25.855 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already out. 2026-02-26T03:02:25.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:25.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.2 2026-02-26T03:02:26.121 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already out. 2026-02-26T03:02:26.130 INFO:tasks.workunit.client.0.trial123.stdout:bringing up non_divergent 0 2 2026-02-26T03:02:26.131 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:514: TEST_divergent_2: echo 'bringing up non_divergent 0 2' 2026-02-26T03:02:26.131 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:515: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:02:26.340 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:02:26.340 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 stale+active+clean 2026-02-26T03:01:59.625423+0000 27'106 27:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:14:55.773702+0000 0 0 2026-02-26T03:02:26.340 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:02:26.340 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:02:26.341 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:02:26.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:26.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 0 2026-02-26T03:02:26.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:02:26.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:02:26.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:02:26.353 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:02:26.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:02:26.355 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:02:26.356 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:02:26.356 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:02:26.356 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-26T03:02:26.356 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:02:26.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:02:26.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:02:26.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:02:26.377 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:26.372+0000 7fe2309938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:02:26.378 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:26.376+0000 7fe2309938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:02:26.379 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:26.376+0000 7fe2309938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:02:26.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:02:26.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:02:26.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:02:26.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:02:26.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:02:26.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:26.563 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:02:26.564 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:02:26.564 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:26.564 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:02:26.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:27.621 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:27.616+0000 7fe2309938c0 -1 Falling back to public interface 2026-02-26T03:02:27.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:27.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:27.767 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:02:27.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:02:27.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:27.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:02:27.966 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:28.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:28.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:28.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:02:28.968 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:02:28.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:28.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:02:29.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:30.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:30.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:30.170 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:02:30.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:02:30.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:30.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:02:30.371 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:30.651 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:30.648+0000 7fe2309938c0 -1 osd.0 27 log_to_monitors true 2026-02-26T03:02:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:31.372 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:02:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:02:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:02:31.570 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up out weight 0 up_from 34 up_thru 0 down_at 28 last_clean_interval [5,27) [v2:127.0.0.1:6802/3873737674,v1:127.0.0.1:6803/3873737674] [v2:127.0.0.1:6804/3873737674,v1:127.0.0.1:6805/3873737674] exists,up 9494153f-5bc6-4a02-8f18-4660dcead5ca 2026-02-26T03:02:31.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:02:31.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 2 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:02:31.573 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:02:31.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:02:31.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:02:31.575 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:02:31.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:02:31.576 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-26T03:02:31.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:02:31.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:02:31.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:02:31.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:02:31.594 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:31.588+0000 7fb5711b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:02:31.595 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:31.592+0000 7fb5711b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:02:31.596 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:31.592+0000 7fb5711b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:31.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:02:31.981 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:32.849 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:32.844+0000 7fb5711b28c0 -1 Falling back to public interface 2026-02-26T03:02:32.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:32.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:32.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:02:32.984 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:02:32.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:32.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:02:33.193 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:34.195 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:34.195 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:34.196 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:02:34.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:02:34.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:34.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:02:34.397 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:35.399 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:35.399 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:35.399 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:02:35.400 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:02:35.400 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:35.400 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:02:35.596 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:02:35.908 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:02:35.904+0000 7fb5711b28c0 -1 osd.2 27 log_to_monitors true 2026-02-26T03:02:36.598 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:02:36.598 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:02:36.598 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:02:36.598 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:02:36.599 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:02:36.599 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:02:36.795 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up out weight 0 up_from 37 up_thru 0 down_at 30 last_clean_interval [14,27) [v2:127.0.0.1:6810/3645775947,v1:127.0.0.1:6811/3645775947] [v2:127.0.0.1:6812/3645775947,v1:127.0.0.1:6813/3645775947] exists,up a75b1a3a-0132-4f1c-babe-e4d4ec680aec 2026-02-26T03:02:36.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:02:36.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:02:36.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:02:36.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:36.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.0 2026-02-26T03:02:37.043 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already in. 2026-02-26T03:02:37.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:02:37.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.2 2026-02-26T03:02:37.314 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already in. 2026-02-26T03:02:37.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: wait_for_clean 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:02:37.324 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:02:37.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:02:37.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:02:37.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:02:37.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:02:37.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:02:37.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:02:37.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:02:37.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:02:37.390 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:02:37.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:02:37.590 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:02:37.590 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:02:37.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:02:37.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:37.590 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:02:37.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888068 2026-02-26T03:02:37.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888068 2026-02-26T03:02:37.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888068' 2026-02-26T03:02:37.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:37.670 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:02:37.729 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:02:37.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:02:37.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:02:37.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:02:37.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:02:37.731 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:02:37.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789955 2026-02-26T03:02:37.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789955 2026-02-26T03:02:37.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888068 2-158913789955' 2026-02-26T03:02:37.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:37.807 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888068 2026-02-26T03:02:37.808 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:37.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:02:37.810 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888068 2026-02-26T03:02:37.810 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:37.811 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888068 2026-02-26T03:02:37.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888068' 2026-02-26T03:02:37.812 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 146028888068 2026-02-26T03:02:37.812 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:02:38.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-26T03:02:38.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:02:38.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-158913789955 2026-02-26T03:02:38.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:02:38.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:02:38.022 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-158913789955 2026-02-26T03:02:38.023 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:02:38.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789955 2026-02-26T03:02:38.024 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 158913789955 2026-02-26T03:02:38.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 158913789955' 2026-02-26T03:02:38.025 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:02:38.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-26T03:02:38.222 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:02:38.222 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:38.222 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:38.478 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:02:38.478 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:38.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:38.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:38.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:38.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:38.479 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:38.479 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:38.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:38.670 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:38.670 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:38.671 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:38.925 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:38.925 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:02:38.925 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:02:38.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:02:38.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:38.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:02:39.026 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:39.026 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:39.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:39.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:39.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:39.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:39.027 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:39.027 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:39.218 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:39.218 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:39.219 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:39.219 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:39.480 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:39.480 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:39.481 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:39.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:39.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:39.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:02:39.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:39.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:02:39.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:39.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:39.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:39.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:39.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:39.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:39.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:39.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:40.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:40.133 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:40.134 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:40.134 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:40.388 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:40.388 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:40.389 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:40.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:40.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:40.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:02:40.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:40.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:02:41.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:41.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:41.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:41.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:41.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:41.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:41.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:41.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:41.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:41.242 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:41.242 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:41.242 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:41.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:41.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:41.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:41.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:41.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:41.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:41.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:41.497 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:41.497 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:41.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:41.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:41.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:02:41.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:41.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:02:42.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:42.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:42.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:42.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:42.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:42.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:42.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:42.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:42.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:42.760 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:42.760 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:42.760 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:43.023 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:43.023 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:43.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:43.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:43.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:02:43.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:43.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:02:44.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:44.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:44.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:44.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:44.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:44.883 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:44.883 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:44.883 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:45.077 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:45.077 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:45.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:45.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:45.336 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:45.337 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:02:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:02:48.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:48.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:48.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:48.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:48.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:48.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:48.798 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:48.799 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:48.995 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:48.995 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:48.996 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:48.996 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:49.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:49.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:49.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:49.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:49.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:49.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:49.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:49.252 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:49.252 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:49.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:49.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:49.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:02:49.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:49.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:02:55.917 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:02:55.918 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:02:55.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:02:55.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:02:55.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:02:55.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:02:55.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:02:55.920 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:02:56.118 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:02:56.119 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:02:56.119 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:02:56.119 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:02:56.382 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:02:56.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:02:56.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:02:56.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:02:56.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:02:56.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:03:03.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:03.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:03.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:04.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:04.138 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:04.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:04.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:04.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:04.393 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:04.393 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:04.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:04.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:04.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:03:04.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "3aba4ffa-2498-47fe-85d8-15f7cecf4852", 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:03:04.900241+0000", 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:03:04.900 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:04.901 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:01:31.262388Z", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:01:31.262388Z", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:03:04.902 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 40, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:01:31.349181+0000", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:02:37.366907+0000", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:02:36.085330+0000", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:02:37.264658+0000", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:03:04.903 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 9, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T03:01:59.427800+0000", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 0, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "off", 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 1, 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:03:04.904 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "27", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "0", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "selfmanaged", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [], 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "", 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:03:04.905 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 0, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {}, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "read_balance": { 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "score_type": "Fair distribution", 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "score_acting": 2.9900000095367432, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "score_stable": 2.9900000095367432, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_score": 0.67000001668930054, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_acting": 2, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_stable": 2, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity_weighted": 1, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity": 1, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity_weighted": 1 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "9494153f-5bc6-4a02-8f18-4660dcead5ca", 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 5, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 27, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 34, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 39, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 28, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.906 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.907 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/3873737674", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/3873737674", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3873737674", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3873737674", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "9554fde6-99f6-48e4-b715-1384b74549a4", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 9, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 21, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 29, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:03:04.908 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3656840459", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3656840459", 2026-02-26T03:03:04.909 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3656840459", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3656840459", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "a75b1a3a-0132-4f1c-babe-e4d4ec680aec", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 14, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 27, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 37, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.910 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3645775947", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3645775947", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3645775947", 2026-02-26T03:03:04.911 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3645775947", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:02:25.016903+0000", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:01:40.165965+0000", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:02:25.279581+0000", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:01:48.376952+0000", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:02:25.548287+0000", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:01:57.744738+0000", 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:03:04.912 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/3873737674,v1:127.0.0.1:6805/3873737674]", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:03:04.913 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:01:33.800550Z", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/3873737674,v1:127.0.0.1:6803/3873737674]", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3873737674,v1:127.0.0.1:6809/3873737674]", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3873737674,v1:127.0.0.1:6807/3873737674]", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:03:04.914 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3656840459,v1:127.0.0.1:6813/3656840459]", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:01:41.841591Z", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3656840459,v1:127.0.0.1:6811/3656840459]", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3656840459,v1:127.0.0.1:6817/3656840459]", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3656840459,v1:127.0.0.1:6815/3656840459]", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:03:04.915 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3645775947,v1:127.0.0.1:6813/3645775947]", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:03:04.916 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:01:51.104821Z", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3645775947,v1:127.0.0.1:6811/3645775947]", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3645775947,v1:127.0.0.1:6817/3645775947]", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3645775947,v1:127.0.0.1:6815/3645775947]", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 0, 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [] 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 34 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 37 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.917 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 40, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.918 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:04.919 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:04.920 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:03:04.921 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 0, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:03:04.922 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stderr:report 436690297 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T03:01:31:348952+0000", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.923 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 1, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 72800, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 100, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 100, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:03:04.924 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 16, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 16, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "up": 2, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 2, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:03:04.925 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83880, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2072, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488920, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036654080, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2121728, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 485902, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:03:04.926 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83880, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2072, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488920, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036654080, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2121728, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 485902, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 72800, 2026-02-26T03:03:04.927 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 100, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 100, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:04.928 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1220608, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 217422, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 16, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 16, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "up": 2, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 2, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 146028888073 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 38654705678 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 158913789960 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num": 1 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:03:04.929 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 1, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 5, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:02:49.833445+0000", 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "0": { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.930 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 44, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 232, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 327, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: expr 2 + 2 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: objname=existing_4 2026-02-26T03:03:04.931 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: echo 'writing non-divergent object existing_4' 2026-02-26T03:03:04.932 INFO:tasks.workunit.client.0.trial123.stdout:writing non-divergent object existing_4 2026-02-26T03:03:04.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:530: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:05.100 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:05.100 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 27'100 40:130 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:05.100 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:05.100 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:05.100 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:05.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:531: TEST_divergent_2: rados -p test put existing_4 /etc/resolv.conf 2026-02-26T03:03:05.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:03:05.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: wait_for_clean 2026-02-26T03:03:05.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:03:05.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:03:05.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:03:05.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:03:05.139 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:03:05.139 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:03:05.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:03:05.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:03:05.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:03:05.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:03:05.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:03:05.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:03:05.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:03:05.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:03:05.211 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:03:05.412 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:03:05.412 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:03:05.412 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:03:05.412 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:03:05.412 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:03:05.412 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:03:05.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888075 2026-02-26T03:03:05.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888075 2026-02-26T03:03:05.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888075' 2026-02-26T03:03:05.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:03:05.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:03:05.555 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:03:05.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:03:05.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:03:05.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:03:05.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:03:05.557 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:03:05.636 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789962 2026-02-26T03:03:05.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789962 2026-02-26T03:03:05.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888075 2-158913789962' 2026-02-26T03:03:05.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:03:05.637 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888075 2026-02-26T03:03:05.638 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:03:05.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:03:05.639 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888075 2026-02-26T03:03:05.639 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:03:05.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888075 2026-02-26T03:03:05.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888075' 2026-02-26T03:03:05.640 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 146028888075 2026-02-26T03:03:05.641 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:03:05.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888073 -lt 146028888075 2026-02-26T03:03:05.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:03:06.847 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:03:06.847 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:03:07.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888075 -lt 146028888075 2026-02-26T03:03:07.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:03:07.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-158913789962 2026-02-26T03:03:07.044 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:03:07.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:03:07.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-158913789962 2026-02-26T03:03:07.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:03:07.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789962 2026-02-26T03:03:07.047 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 158913789962 2026-02-26T03:03:07.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 158913789962' 2026-02-26T03:03:07.048 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:03:07.260 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789962 -lt 158913789962 2026-02-26T03:03:07.261 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:03:07.261 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:07.261 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:07.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:07.531 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:07.729 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:07.730 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:07.730 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:07.730 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:07.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:07.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:03:07.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:03:07.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:03:07.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:07.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:03:08.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:08.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:08.095 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:08.095 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:08.095 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:08.095 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:08.095 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:08.095 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:08.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:08.295 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:08.296 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:08.296 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:08.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:08.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:08.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:08.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:08.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:08.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:03:08.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:08.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:03:09.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:09.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:09.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:09.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:09.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:09.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:09.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:09.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:09.233 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:09.234 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:09.234 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:09.234 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:09.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:09.498 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:09.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:09.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:09.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:03:09.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:09.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:03:10.162 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:10.162 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:10.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:10.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:10.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:10.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:10.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:10.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:10.363 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:10.363 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:10.364 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:10.364 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:10.636 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:10.636 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:10.636 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:10.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:10.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:10.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:10.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:10.637 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:10.637 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:10.903 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:10.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:10.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:03:10.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:10.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:03:11.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:11.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:11.705 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:11.705 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:11.705 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:11.705 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:11.705 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:11.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:11.908 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:11.908 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:11.909 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:11.909 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:12.165 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:12.166 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:12.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:12.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:12.421 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:03:12.421 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:12.421 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:03:14.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:14.021 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:14.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:14.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:14.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:14.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:14.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:14.022 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:14.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:14.220 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:14.221 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:14.221 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:14.481 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:03:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:03:17.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:17.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:17.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:17.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:17.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:17.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:17.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:17.938 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:18.137 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:18.138 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:18.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:18.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:18.409 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:18.409 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:18.409 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:18.409 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:18.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:18.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:18.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:18.410 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:18.410 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:18.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:18.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:18.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:03:18.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:18.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:03:25.071 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:25.072 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:25.072 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:25.072 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:25.072 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:25.072 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:25.073 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:25.073 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:25.277 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:25.277 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:25.277 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:25.277 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:25.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:25.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:25.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:25.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:25.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:25.547 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:25.548 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:25.548 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:25.548 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:25.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:25.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:25.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:03:25.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:03:25.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:03:33.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:03:33.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:03:33.115 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:03:33.115 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:03:33.115 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:03:33.116 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:03:33.116 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:03:33.116 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:03:33.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:03:33.321 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:03:33.321 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:03:33.321 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:03:33.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:03:33.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:03:33.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:03:33.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:03:33.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:03:33.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:03:33.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:03:33.584 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:03:33.584 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:03:33.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:03:33.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:03:33.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:03:33.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:03:34.099 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:03:34.099 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "3aba4ffa-2498-47fe-85d8-15f7cecf4852", 2026-02-26T03:03:34.099 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:03:34.099285+0000", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:03:34.100 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.101 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-26T03:03:34.102 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:03:34.103 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.104 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:03:34.105 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:01:31.262388Z", 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:01:31.262388Z", 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:03:34.106 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:03:34.107 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:03:34.108 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:03:34.109 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 40, 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:01:31.349181+0000", 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:02:37.366907+0000", 2026-02-26T03:03:34.110 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:02:36.085330+0000", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:02:37.264658+0000", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.111 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 9, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:03:34.112 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T03:01:59.427800+0000", 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 1, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool", 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 0, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:03:34.113 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "off", 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 1, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 1, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 1, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 1, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:03:34.114 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "27", 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "0", 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "selfmanaged", 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 0, 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 0, 2026-02-26T03:03:34.115 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [], 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:03:34.116 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "", 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:03:34.117 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 0, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {}, 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "read_balance": { 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "score_type": "Fair distribution", 2026-02-26T03:03:34.118 INFO:tasks.workunit.client.0.trial123.stdout: "score_acting": 2.9900000095367432, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "score_stable": 2.9900000095367432, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_score": 0.67000001668930054, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_acting": 2, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_stable": 2, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity_weighted": 1, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity": 1, 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity_weighted": 1 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.119 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "9494153f-5bc6-4a02-8f18-4660dcead5ca", 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 5, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 27, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 34, 2026-02-26T03:03:34.120 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 39, 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 28, 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.121 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.122 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.123 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:03:34.124 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3873737674 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.125 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/3873737674", 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/3873737674", 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3873737674", 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3873737674", 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.126 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "9554fde6-99f6-48e4-b715-1384b74549a4", 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 9, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 21, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 29, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:34.127 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.129 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.130 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3656840459 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3656840459", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3656840459", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3656840459", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3656840459", 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:03:34.131 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "a75b1a3a-0132-4f1c-babe-e4d4ec680aec", 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 14, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 27, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 37, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:03:34.132 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.133 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:03:34.135 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3645775947 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3645775947", 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3645775947", 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3645775947", 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3645775947", 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:02:25.016903+0000", 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:03:34.136 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:01:40.165965+0000", 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:02:25.279581+0000", 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:01:48.376952+0000", 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:02:25.548287+0000", 2026-02-26T03:03:34.137 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:01:57.744738+0000", 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:03:34.138 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/3873737674,v1:127.0.0.1:6805/3873737674]", 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:03:34.139 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:03:34.140 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:01:33.800550Z", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/3873737674,v1:127.0.0.1:6803/3873737674]", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3873737674,v1:127.0.0.1:6809/3873737674]", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3873737674,v1:127.0.0.1:6807/3873737674]", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:03:34.141 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3656840459,v1:127.0.0.1:6813/3656840459]", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:03:34.142 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:01:41.841591Z", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:03:34.143 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3656840459,v1:127.0.0.1:6811/3656840459]", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3656840459,v1:127.0.0.1:6817/3656840459]", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3656840459,v1:127.0.0.1:6815/3656840459]", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.144 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3645775947,v1:127.0.0.1:6813/3645775947]", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:03:34.145 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:01:51.104821Z", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3645775947,v1:127.0.0.1:6811/3645775947]", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3645775947,v1:127.0.0.1:6817/3645775947]", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3645775947,v1:127.0.0.1:6815/3645775947]", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:03:34.146 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 0, 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [] 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 34 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 37 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.147 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 40, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.148 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:03:34.149 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:03:34.150 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:03:34.151 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.152 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.153 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:03:34.154 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 0, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:03:34.155 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stderr:report 4277915698 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T03:01:31:348952+0000", 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:03:34.156 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 1, 2026-02-26T03:03:34.157 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73039, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:03:34.158 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 17, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 17, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "up": 2, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 2, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.159 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83880, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2072, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488920, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036654080, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2121728, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 486380, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.160 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83880, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2072, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488920, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036654080, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2121728, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 486380, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:03:34.161 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73039, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:03:34.162 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.163 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1220608, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 217900, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 17, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 17, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "up": 2, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 2, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 146028888081 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 38654705678 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 158913789968 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.164 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num": 1 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 1, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:03:34.165 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 6, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:03:31.838375+0000", 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 59, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 284, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 394, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:03:34.166 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:03:34.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:03:34.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:536: TEST_divergent_2: echo 'delay recovery' 2026-02-26T03:03:34.167 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery 2026-02-26T03:03:34.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:537: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:34.306 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:34.306 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 40:131 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:34.306 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:34.306 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:34.306 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:34.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:03:34.315 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.0 2026-02-26T03:03:34.315 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:03:34.315 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:03:34.316 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:03:34.316 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:03:34.316 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:03:34.316 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:03:34.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2026-02-26T03:03:34.317 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok set_recovery_delay 100000 2026-02-26T03:03:34.370 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:03:34.376 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2026-02-26T03:03:34.377 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.2 2026-02-26T03:03:34.377 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:03:34.377 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:03:34.377 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:03:34.377 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:03:34.377 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:03:34.378 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:03:34.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2026-02-26T03:03:34.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok set_recovery_delay 100000 2026-02-26T03:03:34.429 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:03:34.434 INFO:tasks.workunit.client.0.trial123.stdout:revive divergent 1 2026-02-26T03:03:34.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:544: TEST_divergent_2: echo 'revive divergent 1' 2026-02-26T03:03:34.435 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:545: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:34.633 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:34.633 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 40:131 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:34.634 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:34.634 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:34.634 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:34.642 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:546: TEST_divergent_2: ceph osd set noup 2026-02-26T03:03:34.957 INFO:tasks.workunit.client.0.trial123.stderr:noup is set 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:547: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:03:34.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:03:34.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:03:34.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:03:34.971 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:03:34.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:03:34.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:03:34.972 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:03:34.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:03:34.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:03:34.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:03:34.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:03:34.995 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:34.994+0000 7fd0d9cea8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:03:34.996 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:34.994+0000 7fd0d9cea8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:03:34.997 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:34.994+0000 7fd0d9cea8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:03:35.175 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-26T03:03:35.175 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:548: TEST_divergent_2: sleep 5 2026-02-26T03:03:36.236 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:36.234+0000 7fd0d9cea8c0 -1 Falling back to public interface 2026-02-26T03:03:39.235 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:39.235+0000 7fd0d9cea8c0 -1 osd.1 27 log_to_monitors true 2026-02-26T03:03:40.176 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery divergent 2026-02-26T03:03:40.177 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:550: TEST_divergent_2: echo 'delay recovery divergent' 2026-02-26T03:03:40.177 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:551: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:40.367 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:40.367 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 42:134 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:40.368 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:40.368 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:40.368 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:40.377 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: get_asok_path osd.1 2026-02-26T03:03:40.377 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:03:40.377 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:03:40.377 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:03:40.377 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:03:40.377 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:03:40.378 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:03:40.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: CEPH_ARGS= 2026-02-26T03:03:40.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok set_recovery_delay 100000 2026-02-26T03:03:40.429 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:03:40.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: ceph osd unset noup 2026-02-26T03:03:40.746 INFO:tasks.workunit.client.0.trial123.stderr:noup is unset 2026-02-26T03:03:40.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:556: TEST_divergent_2: wait_for_osd up 0 2026-02-26T03:03:40.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:03:40.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:03:40.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:03:40.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:03:40.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:03:40.759 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:03:40.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:03:40.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:03:40.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:03:40.956 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 34 up_thru 39 down_at 28 last_clean_interval [5,27) [v2:127.0.0.1:6802/3873737674,v1:127.0.0.1:6803/3873737674] [v2:127.0.0.1:6804/3873737674,v1:127.0.0.1:6805/3873737674] exists,up 9494153f-5bc6-4a02-8f18-4660dcead5ca 2026-02-26T03:03:40.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:03:40.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:03:40.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:03:40.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:557: TEST_divergent_2: wait_for_osd up 1 2026-02-26T03:03:40.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:03:40.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:03:40.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:03:40.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:03:40.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:03:40.958 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:03:40.959 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:03:40.959 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:03:40.959 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:03:41.152 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 29 last_clean_interval [9,27) [v2:127.0.0.1:6818/571568380,v1:127.0.0.1:6819/571568380] [v2:127.0.0.1:6820/571568380,v1:127.0.0.1:6821/571568380] exists,up 9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:03:41.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:03:41.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:03:41.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:03:41.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:558: TEST_divergent_2: wait_for_osd up 2 2026-02-26T03:03:41.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:03:41.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:03:41.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:03:41.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:03:41.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:03:41.154 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:03:41.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:03:41.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:03:41.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:03:41.353 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 37 up_thru 0 down_at 30 last_clean_interval [14,27) [v2:127.0.0.1:6810/3645775947,v1:127.0.0.1:6811/3645775947] [v2:127.0.0.1:6812/3645775947,v1:127.0.0.1:6813/3645775947] exists,up a75b1a3a-0132-4f1c-babe-e4d4ec680aec 2026-02-26T03:03:41.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:03:41.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:03:41.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:03:41.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:560: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:41.544 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:41.544 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 42:134 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:41.544 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:41.545 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:41.545 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:41.554 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:561: TEST_divergent_2: echo 'wait for peering' 2026-02-26T03:03:41.555 INFO:tasks.workunit.client.0.trial123.stdout:wait for peering 2026-02-26T03:03:41.555 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:562: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:41.739 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:41.739 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 42:134 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:41.739 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:41.739 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:41.739 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:41.748 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:563: TEST_divergent_2: rados -p test put foo /etc/fstab 2026-02-26T03:03:41.777 INFO:tasks.workunit.client.0.trial123.stdout:killing divergent 1 2026-02-26T03:03:41.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:567: TEST_divergent_2: echo 'killing divergent 1' 2026-02-26T03:03:41.778 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:568: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:41.960 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:41.960 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 42:134 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:41.960 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:41.960 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:41.960 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:41.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:569: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd.1 2026-02-26T03:03:41.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:03:41.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:03:41.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:03:41.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:03:41.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:03:42.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:03:42.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:572: TEST_divergent_2: expfile=td/divergent-priors/exp.18886.out 2026-02-26T03:03:42.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:573: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.18886.out 2026-02-26T03:03:42.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:03:42.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:03:42.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-26T03:03:42.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:03:42.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-26T03:03:42.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.18886.out 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Exporting 1.0 info 1.0( v 46'102 lc 0'0 (27'90,46'102] local-lis/les=45/46 n=101 ec=21/21 lis/c=45/21 les/c/f=46/22/0 sis=45) 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:091f9f1d:::existing_94:head# 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:0a865a93:::existing_54:head# 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:0fed513c:::existing_41:head# 2026-02-26T03:03:42.755 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:13b9e0b8:::existing_92:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:169157fa:::existing_81:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:17731382:::existing_58:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:18a45a39:::existing_43:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:1c160f01:::existing_66:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:1dd6c249:::existing_97:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:227e0c77:::existing_7:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:23e1b063:::existing_59:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:28566edb:::existing_78:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:2a101182:::existing_20:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:2d1519da:::existing_96:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:3078b87a:::existing_86:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:30bde01b:::existing_15:head# 2026-02-26T03:03:42.756 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:317a7e19:::existing_56:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:38b9c7fa:::existing_33:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:3e16d84b:::existing_48:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:45fe8203:::existing_75:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:4e82c9ba:::existing_50:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:5298dcdb:::existing_57:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:52ebaa44:::existing_32:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:53588912:::existing_9:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:5b7e2de6:::existing_84:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:5da0255c:::existing_13:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:5da2dab8:::existing_53:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:5ee3514b:::existing_30:head# 2026-02-26T03:03:42.757 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:602f83fe:::foo:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:60426cc7:::existing_62:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:628918ca:::existing_79:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:63f8ab2d:::existing_77:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:67e1780f:::existing_27:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:68143ca1:::existing_71:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:6bac9509:::existing_29:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:6bbec798:::existing_45:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:6cb23a40:::existing_65:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:7055ec5e:::existing_83:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:70a54e2b:::existing_87:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:750ace21:::existing_19:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:78623b1d:::existing_6:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:79881152:::existing_63:head# 2026-02-26T03:03:42.758 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:7a80c0d5:::existing_99:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:81490677:::existing_76:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:85ec8e5d:::existing_85:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:8e2abbf0:::existing_49:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:8ee0fe0f:::existing_52:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:923c5d4d:::existing_5:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:92731e71:::existing_89:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:928205cf:::existing_82:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:93be2267:::existing_24:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:99ceda2c:::existing_44:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:9dd917ee:::existing_26:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a28805a6:::existing_72:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a294a8f6:::existing_16:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a2f5473a:::existing_42:head# 2026-02-26T03:03:42.759 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a504a533:::existing_90:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a691337e:::existing_61:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a6e8d7ac:::existing_28:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:a9fde86e:::existing_18:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:aa309faa:::existing_68:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:ae40b0ed:::existing_31:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:b279b1fe:::existing_10:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:b343e277:::existing_88:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:b7130e90:::existing_80:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:bc6f7309:::existing_70:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:be8489b7:::existing_38:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:beb4e5e3:::existing_47:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:bebec877:::existing_74:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:c04347d8:::existing_60:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:c156033e:::existing_100:head# 2026-02-26T03:03:42.760 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:c19f5a8e:::existing_8:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:c3e6ebe4:::existing_17:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:c6c5a7c2:::existing_39:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:cc1d0abf:::existing_95:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:ccb3865b:::existing_73:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:cd84e19e:::existing_40:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:d2cd64ce:::existing_37:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:d41a25c9:::existing_22:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:d7f088da:::existing_91:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:d9cbd1d6:::existing_64:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:dcc4ff82:::existing_98:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:dd677bd3:::existing_35:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:dd6d90e7:::existing_46:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:e0e22077:::existing_21:head# 2026-02-26T03:03:42.761 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:e255b710:::existing_55:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:eff1d5db:::existing_14:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:f1b156bf:::existing_23:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:f5ce4199:::existing_34:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:f6809456:::existing_11:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:f699a2a9:::existing_67:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:f9baac43:::existing_25:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:fb76688e:::existing_51:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:fbb92eec:::existing_69:head# 2026-02-26T03:03:42.762 INFO:tasks.workunit.client.0.trial123.stderr:Export successful 2026-02-26T03:03:42.763 INFO:tasks.workunit.client.0.trial123.stdout: marking collection for removal 2026-02-26T03:03:42.763 INFO:tasks.workunit.client.0.trial123.stdout:setting '_remove' omap key 2026-02-26T03:03:42.763 INFO:tasks.workunit.client.0.trial123.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-26T03:03:42.763 INFO:tasks.workunit.client.0.trial123.stderr:Remove successful 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:574: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.18886.out 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-26T03:03:43.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.18886.out 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:get_pg_num_history pg_num_history pg_num_history(e46 pg_nums {1={21=1}} deleted_pools ) 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:Importing pgid 1.0 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-26T03:03:43.917 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.918 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:091f9f1d:::existing_94:head# 2026-02-26T03:03:43.918 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.918 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:0a865a93:::existing_54:head# 2026-02-26T03:03:43.918 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.918 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:0fed513c:::existing_41:head# 2026-02-26T03:03:43.918 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:13b9e0b8:::existing_92:head# 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:169157fa:::existing_81:head# 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:17731382:::existing_58:head# 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:18a45a39:::existing_43:head# 2026-02-26T03:03:43.919 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:1c160f01:::existing_66:head# 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:1dd6c249:::existing_97:head# 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:227e0c77:::existing_7:head# 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.920 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:23e1b063:::existing_59:head# 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:28566edb:::existing_78:head# 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:2a101182:::existing_20:head# 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:2d1519da:::existing_96:head# 2026-02-26T03:03:43.921 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:3078b87a:::existing_86:head# 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:30bde01b:::existing_15:head# 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:317a7e19:::existing_56:head# 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:38b9c7fa:::existing_33:head# 2026-02-26T03:03:43.922 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:3e16d84b:::existing_48:head# 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:45fe8203:::existing_75:head# 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:4e82c9ba:::existing_50:head# 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.923 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:5298dcdb:::existing_57:head# 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:52ebaa44:::existing_32:head# 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:53588912:::existing_9:head# 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:5b7e2de6:::existing_84:head# 2026-02-26T03:03:43.924 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:5da0255c:::existing_13:head# 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:5da2dab8:::existing_53:head# 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:5ee3514b:::existing_30:head# 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:602f83fe:::foo:head# 2026-02-26T03:03:43.925 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:60426cc7:::existing_62:head# 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:628918ca:::existing_79:head# 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:63f8ab2d:::existing_77:head# 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:67e1780f:::existing_27:head# 2026-02-26T03:03:43.926 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.927 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:68143ca1:::existing_71:head# 2026-02-26T03:03:43.927 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.927 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:6bac9509:::existing_29:head# 2026-02-26T03:03:43.927 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.927 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:6bbec798:::existing_45:head# 2026-02-26T03:03:43.927 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:6cb23a40:::existing_65:head# 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:7055ec5e:::existing_83:head# 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:70a54e2b:::existing_87:head# 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:750ace21:::existing_19:head# 2026-02-26T03:03:43.928 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:78623b1d:::existing_6:head# 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:79881152:::existing_63:head# 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:7a80c0d5:::existing_99:head# 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:81490677:::existing_76:head# 2026-02-26T03:03:43.929 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.930 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:85ec8e5d:::existing_85:head# 2026-02-26T03:03:43.930 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.930 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:8e2abbf0:::existing_49:head# 2026-02-26T03:03:43.930 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.930 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:8ee0fe0f:::existing_52:head# 2026-02-26T03:03:43.930 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.931 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:923c5d4d:::existing_5:head# 2026-02-26T03:03:43.931 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.931 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:92731e71:::existing_89:head# 2026-02-26T03:03:43.931 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.932 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:928205cf:::existing_82:head# 2026-02-26T03:03:43.932 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.935 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:93be2267:::existing_24:head# 2026-02-26T03:03:43.935 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.935 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:99ceda2c:::existing_44:head# 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:9dd917ee:::existing_26:head# 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a28805a6:::existing_72:head# 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a294a8f6:::existing_16:head# 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.936 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a2f5473a:::existing_42:head# 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a504a533:::existing_90:head# 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a691337e:::existing_61:head# 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a6e8d7ac:::existing_28:head# 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:a9fde86e:::existing_18:head# 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.937 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:aa309faa:::existing_68:head# 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:ae40b0ed:::existing_31:head# 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:b279b1fe:::existing_10:head# 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:b343e277:::existing_88:head# 2026-02-26T03:03:43.938 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:b7130e90:::existing_80:head# 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:bc6f7309:::existing_70:head# 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:be8489b7:::existing_38:head# 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.945 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:beb4e5e3:::existing_47:head# 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:bebec877:::existing_74:head# 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:c04347d8:::existing_60:head# 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:c156033e:::existing_100:head# 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:c19f5a8e:::existing_8:head# 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:c3e6ebe4:::existing_17:head# 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.946 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:c6c5a7c2:::existing_39:head# 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:cc1d0abf:::existing_95:head# 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:ccb3865b:::existing_73:head# 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:cd84e19e:::existing_40:head# 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:d2cd64ce:::existing_37:head# 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:d41a25c9:::existing_22:head# 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.947 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:d7f088da:::existing_91:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:d9cbd1d6:::existing_64:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:dcc4ff82:::existing_98:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:dd677bd3:::existing_35:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:dd6d90e7:::existing_46:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:e0e22077:::existing_21:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:e255b710:::existing_55:head# 2026-02-26T03:03:43.948 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:eff1d5db:::existing_14:head# 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:f1b156bf:::existing_23:head# 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:f5ce4199:::existing_34:head# 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:f6809456:::existing_11:head# 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:f699a2a9:::existing_67:head# 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:f9baac43:::existing_25:head# 2026-02-26T03:03:43.949 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.950 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:fb76688e:::existing_51:head# 2026-02-26T03:03:43.950 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.950 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:fbb92eec:::existing_69:head# 2026-02-26T03:03:43.950 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:03:43.950 INFO:tasks.workunit.client.0.trial123.stdout:write_pg epoch 46 info 1.0( v 46'102 lc 0'0 (27'90,46'102] local-lis/les=45/46 n=101 ec=21/21 lis/c=45/21 les/c/f=46/22/0 sis=45) 2026-02-26T03:03:43.950 INFO:tasks.workunit.client.0.trial123.stdout:Import successful 2026-02-26T03:03:44.484 INFO:tasks.workunit.client.0.trial123.stdout:reviving divergent 1 2026-02-26T03:03:44.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:576: TEST_divergent_2: echo 'reviving divergent 1' 2026-02-26T03:03:44.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:577: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:03:44.675 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:03:44.675 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:02:37.371238+0000 40'101 42:134 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:54:14.183623+0000 0 0 2026-02-26T03:03:44.675 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:03:44.675 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:03:44.675 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:578: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:03:44.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:03:44.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:03:44.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:03:44.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:03:44.685 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:03:44.686 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:03:44.686 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:03:44.686 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:03:44.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:03:44.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:03:44.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:03:44.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:03:44.688 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:03:44.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:03:44.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:03:44.689 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:03:44.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:03:44.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:03:44.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:03:44.690 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:03:44.713 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:44.711+0000 7f6a8c4698c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:03:44.714 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:44.715+0000 7f6a8c4698c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:03:44.715 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:44.715+0000 7f6a8c4698c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:03:44.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:03:44.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:03:44.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:03:44.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:03:44.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:03:44.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:03:44.886 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:03:44.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:03:44.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:03:44.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:03:45.088 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 29 last_clean_interval [9,27) [v2:127.0.0.1:6818/571568380,v1:127.0.0.1:6819/571568380] [v2:127.0.0.1:6820/571568380,v1:127.0.0.1:6821/571568380] exists,up 9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:03:45.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:03:45.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:03:45.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:03:45.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:579: TEST_divergent_2: wait_for_osd up 1 2026-02-26T03:03:45.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:03:45.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:03:45.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:03:45.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:03:45.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:03:45.090 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:03:45.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:03:45.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:03:45.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:03:45.283 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 29 last_clean_interval [9,27) [v2:127.0.0.1:6818/571568380,v1:127.0.0.1:6819/571568380] [v2:127.0.0.1:6820/571568380,v1:127.0.0.1:6821/571568380] exists,up 9554fde6-99f6-48e4-b715-1384b74549a4 2026-02-26T03:03:45.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:03:45.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:03:45.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:03:45.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:581: TEST_divergent_2: sleep 20 2026-02-26T03:03:46.048 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:46.047+0000 7f6a8c4698c0 -1 Falling back to public interface 2026-02-26T03:03:49.059 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:03:49.059+0000 7f6a8c4698c0 -1 osd.1 46 log_to_monitors true 2026-02-26T03:04:05.287 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: get_asok_path osd.1 2026-02-26T03:04:05.288 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:04:05.288 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:04:05.288 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:04:05.288 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:05.288 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:05.289 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:04:05.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: CEPH_ARGS= 2026-02-26T03:04:05.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok dump_ops_in_flight 2026-02-26T03:04:05.342 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:04:05.342 INFO:tasks.workunit.client.0.trial123.stdout: "ops": [], 2026-02-26T03:04:05.342 INFO:tasks.workunit.client.0.trial123.stdout: "num_ops": 0 2026-02-26T03:04:05.342 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:04:05.348 INFO:tasks.workunit.client.0.trial123.stdout:allowing recovery 2026-02-26T03:04:05.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:584: TEST_divergent_2: echo 'allowing recovery' 2026-02-26T03:04:05.348 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:04:05.555 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:04:05.555 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 0 0 0 73767 0 0 12 90 12 active+clean 2026-02-26T03:03:50.290524+0000 46'102 50:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:51:11.050723+0000 0 0 2026-02-26T03:04:05.555 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:04:05.555 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:04:05.556 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:04:05.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-26T03:04:05.564 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-26T03:04:05.648 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:04:05.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-26T03:04:05.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-26T03:04:05.732 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:04:05.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-26T03:04:05.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-26T03:04:05.815 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:04:05.825 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: echo 'reading divergent objects' 2026-02-26T03:04:05.825 INFO:tasks.workunit.client.0.trial123.stdout:reading divergent objects 2026-02-26T03:04:05.825 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:593: TEST_divergent_2: ceph pg dump pgs 2026-02-26T03:04:06.023 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:04:06.023 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 0 0 0 73767 0 0 12 90 12 active+clean 2026-02-26T03:03:50.290524+0000 46'102 50:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:01:59.461172+0000 0'0 2026-02-26T03:01:59.461172+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:51:11.050723+0000 0 0 2026-02-26T03:04:06.023 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:04:06.023 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:04:06.024 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:04:06.032 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: expr 2 + 2 2026-02-26T03:04:06.033 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: seq 1 4 2026-02-26T03:04:06.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:04:06.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_1 td/divergent-priors/existing 2026-02-26T03:04:06.056 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:04:06.056 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_2 td/divergent-priors/existing 2026-02-26T03:04:06.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:04:06.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_3 td/divergent-priors/existing 2026-02-26T03:04:06.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:04:06.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_4 td/divergent-priors/existing 2026-02-26T03:04:06.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: seq 1 2 2026-02-26T03:04:06.140 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:04:06.141 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-26T03:04:06.159 INFO:tasks.workunit.client.0.trial123.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-26T03:04:06.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:04:06.161 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-26T03:04:06.187 INFO:tasks.workunit.client.0.trial123.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-26T03:04:06.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:602: TEST_divergent_2: rm -f td/divergent-priors/existing 2026-02-26T03:04:06.190 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-26T03:04:06.192 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:04:06.195 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 27'106 (27'4) delete 1:70b3bf12:::existing_4:head by client.4567.0:1 2026-02-26T03:02:14.464253+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:04:06.195 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: keeping 27'106 (27'4) delete 1:70b3bf12:::existing_4:head by client.4567.0:1 2026-02-26T03:02:14.464253+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:04:06.195 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:04:06.195 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 27'4 first_divergent_update: 27'106 last_divergent_update: 27'106 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: more recent entry found: 40'101 (27'4) modify 1:70b3bf12:::existing_4:head by client.4686.0:1 2026-02-26T03:03:05.135110+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false, already merged 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 27'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4554.0:1 2026-02-26T03:02:14.455807+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: keeping 27'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4554.0:1 2026-02-26T03:02:14.455807+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 27'101 last_divergent_update: 27'101 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 27'105 (27'3) delete 1:8881e6d8:::existing_3:head by client.4564.0:1 2026-02-26T03:02:14.461814+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: keeping 27'105 (27'3) delete 1:8881e6d8:::existing_3:head by client.4564.0:1 2026-02-26T03:02:14.461814+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:04:06.196 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 27'3 first_divergent_update: 27'105 last_divergent_update: 27'105 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 27'106 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 27'105 (27'3) delete 1:8881e6d8:::existing_3:head by client.4564.0:1 2026-02-26T03:02:14.461814+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 27'3 <= info.log_tail 27'90 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 27'104 (27'2) modify 1:97db2a0f:::existing_2:head by client.4562.0:1 2026-02-26T03:02:14.460333+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: keeping 27'104 (27'2) modify 1:97db2a0f:::existing_2:head by client.4562.0:1 2026-02-26T03:02:14.460333+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 27'2 first_divergent_update: 27'104 last_divergent_update: 27'104 2026-02-26T03:04:06.197 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 27'106 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 27'104 (27'2) modify 1:97db2a0f:::existing_2:head by client.4562.0:1 2026-02-26T03:02:14.460333+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 27'2 <= info.log_tail 27'90 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 27'103 (27'1) modify 1:d858c035:::existing_1:head by client.4559.0:1 2026-02-26T03:02:14.458662+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:04:06.198 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: keeping 27'103 (27'1) modify 1:d858c035:::existing_1:head by client.4559.0:1 2026-02-26T03:02:14.458662+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 27'1 first_divergent_update: 27'103 last_divergent_update: 27'103 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 27'106 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 27'103 (27'1) modify 1:d858c035:::existing_1:head by client.4559.0:1 2026-02-26T03:02:14.458662+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:04:06.199 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 27'1 <= info.log_tail 27'90 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 27'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4556.0:1 2026-02-26T03:02:14.456922+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 20 _merge_object_divergent_entries: keeping 27'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4556.0:1 2026-02-26T03:02:14.456922+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 27'102 last_divergent_update: 27'102 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-26T03:04:06.200 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:03:40.811+0000 7fd0bbc70640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-26T03:04:06.201 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-26T03:04:06.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:04:06.201 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-26T03:04:06.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:04:06.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:617: TEST_divergent_2: echo success 2026-02-26T03:04:06.206 INFO:tasks.workunit.client.0.trial123.stdout:success 2026-02-26T03:04:06.206 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:619: TEST_divergent_2: rm td/divergent-priors/td/divergent-priors/exp.18886.out 2026-02-26T03:04:06.207 INFO:tasks.workunit.client.0.trial123.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.18886.out': No such file or directory 2026-02-26T03:04:06.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:621: TEST_divergent_2: delete_pool test 2026-02-26T03:04:06.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-26T03:04:06.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-26T03:04:06.454 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' does not exist 2026-02-26T03:04:06.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:622: TEST_divergent_2: kill_daemons td/divergent-priors 2026-02-26T03:04:06.465 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:04:06.465 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:04:06.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:04:06.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:04:06.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:04:14.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:04:14.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-26T03:04:14.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:04:14.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:04:14.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:04:14.781 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:04:14.782 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:04:14.782 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:04:14.782 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:04:14.782 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:04:14.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:04:14.789 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:04:14.790 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:04:14.791 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:04:14.792 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:04:14.792 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:04:14.793 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:04:14.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:04:14.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:04:14.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:04:14.795 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:04:14.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:04:14.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:04:14.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:04:14.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:04:14.818 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:14.818 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:14.818 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:04:14.819 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:04:14.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:04:14.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:04:14.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:04:14.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:04:14.822 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:04:14.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:04:14.824 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:04:14.825 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:04:14.825 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:04:14.825 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:04:14.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:04:14.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:04:14.827 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:04:14.827 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:04:14.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:04:14.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:04:14.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:04:14.830 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:04:14.830 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:14.830 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:14.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:04:14.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:04:14.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:04:14.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-26T03:04:14.833 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:04:14.833 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:14.833 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:14.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.18886 2026-02-26T03:04:14.835 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:04:14.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-26T03:04:14.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:04:14.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-26T03:04:14.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_3 td/divergent-priors 2026-02-26T03:04:14.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:628: TEST_divergent_3: local dir=td/divergent-priors 2026-02-26T03:04:14.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:631: TEST_divergent_3: local dummyfile=/etc/fstab 2026-02-26T03:04:14.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: local dummyfile2=/etc/resolv.conf 2026-02-26T03:04:14.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: local num_osds=3 2026-02-26T03:04:14.837 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: expr 3 - 1 2026-02-26T03:04:14.838 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: seq 0 2 2026-02-26T03:04:14.839 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: local 'osds=0 2026-02-26T03:04:14.839 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:636: TEST_divergent_3: run_mon td/divergent-priors a 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-26T03:04:14.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-26T03:04:14.867 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:04:14.868 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:04:14.868 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:04:14.868 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:04:14.868 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:14.869 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:14.869 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:14.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:04:14.901 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:04:14.902 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:04:14.902 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:04:14.902 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:04:14.902 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:04:14.903 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:04:14.903 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:04:14.903 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:04:14.903 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:04:14.903 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:04:14.903 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:14.904 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:14.904 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T03:04:14.904 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:04:14.904 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get fsid 2026-02-26T03:04:14.959 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:04:14.959 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:04:14.959 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:04:14.959 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T03:04:14.961 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:04:14.962 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get mon_host 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: run_mgr td/divergent-priors x 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-26T03:04:15.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:04:15.130 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:04:15.130 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:04:15.131 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:04:15.131 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:04:15.131 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:15.131 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:15.131 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:15.131 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:04:15.132 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:04:15.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-26T03:04:15.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 0 2026-02-26T03:04:15.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:04:15.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:15.154 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:15.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:15.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:04:15.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=03f68cfd-282b-40c9-8b38-71cb31fbe97b 2026-02-26T03:04:15.157 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 03f68cfd-282b-40c9-8b38-71cb31fbe97b 2026-02-26T03:04:15.157 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 03f68cfd-282b-40c9-8b38-71cb31fbe97b' 2026-02-26T03:04:15.157 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:04:15.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAvuJ9paFkzChAAIhRGkzji/tNy1mWJlJf9iA== 2026-02-26T03:04:15.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAvuJ9paFkzChAAIhRGkzji/tNy1mWJlJf9iA=="}' 2026-02-26T03:04:15.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 03f68cfd-282b-40c9-8b38-71cb31fbe97b -i td/divergent-priors/0/new.json 2026-02-26T03:04:15.283 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:04:15.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-26T03:04:15.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAvuJ9paFkzChAAIhRGkzji/tNy1mWJlJf9iA== --osd-uuid 03f68cfd-282b-40c9-8b38-71cb31fbe97b 2026-02-26T03:04:15.310 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:15.311+0000 7f27912938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:15.312 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:15.311+0000 7f27912938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:15.313 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:15.311+0000 7f27912938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:15.313 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:15.311+0000 7f27912938c0 -1 bdev(0x563b6faebc00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:04:15.313 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:15.311+0000 7f27912938c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-26T03:04:18.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-26T03:04:18.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:04:18.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:04:18.099 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:04:18.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:04:18.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:04:18.369 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:04:18.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:04:18.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:04:18.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:04:18.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:04:18.398 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:18.399+0000 7fe10bac28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:18.400 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:18.399+0000 7fe10bac28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:18.401 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:18.399+0000 7fe10bac28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:18.567 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:04:18.567 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:04:18.567 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:04:18.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:04:18.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:04:18.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:18.568 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:04:18.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:04:18.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:18.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:04:18.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:19.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:19.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:19.767 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:04:19.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:04:19.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:19.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:04:19.887 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:19.887+0000 7fe10bac28c0 -1 Falling back to public interface 2026-02-26T03:04:19.964 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:20.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:20.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:20.965 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:04:20.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:04:20.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:20.966 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:04:21.163 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:22.164 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:22.164 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:22.164 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:04:22.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:04:22.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:22.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:04:22.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:22.942 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:22.943+0000 7fe10bac28c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:04:23.358 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:23.358 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:23.358 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:04:23.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:04:23.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:23.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:04:23.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:24.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:24.561 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:24.561 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:04:24.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:04:24.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:24.562 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:04:24.756 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/69412290,v1:127.0.0.1:6803/69412290] [v2:127.0.0.1:6804/69412290,v1:127.0.0.1:6805/69412290] exists,up 03f68cfd-282b-40c9-8b38-71cb31fbe97b 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 1 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:04:24.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:24.759 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:24.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:24.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:04:24.761 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 06feee01-2860-490b-b9ae-8a404b84c218' 2026-02-26T03:04:24.762 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:04:24.762 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:04:24.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA4uJ9p6pBDLhAA8X5lmPTNkGUxRXniTnN0HA== 2026-02-26T03:04:24.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA4uJ9p6pBDLhAA8X5lmPTNkGUxRXniTnN0HA=="}' 2026-02-26T03:04:24.775 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 06feee01-2860-490b-b9ae-8a404b84c218 -i td/divergent-priors/1/new.json 2026-02-26T03:04:24.975 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:04:24.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-26T03:04:24.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA4uJ9p6pBDLhAA8X5lmPTNkGUxRXniTnN0HA== --osd-uuid 06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:04:25.004 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:25.002+0000 7f60666448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:25.005 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:25.006+0000 7f60666448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:25.007 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:25.006+0000 7f60666448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:25.007 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:25.006+0000 7f60666448c0 -1 bdev(0x55f18b845c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-26T03:04:25.007 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:25.006+0000 7f60666448c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-26T03:04:27.299 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-26T03:04:27.299 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:04:27.300 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T03:04:27.300 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T03:04:27.301 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:04:27.572 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:04:27.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T03:04:27.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:04:27.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:04:27.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:04:27.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:04:27.601 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:27.598+0000 7f46963c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:27.603 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:27.602+0000 7f46963c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:27.604 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:27.602+0000 7f46963c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:27.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:04:27.771 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:04:27.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:27.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:04:27.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:28.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:28.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:28.971 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:04:28.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:04:28.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:28.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:04:29.091 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:29.090+0000 7f46963c98c0 -1 Falling back to public interface 2026-02-26T03:04:29.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:30.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:30.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:30.173 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:04:30.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:04:30.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:30.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:04:30.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:31.371 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:31.371 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:31.371 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:04:31.371 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:04:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:31.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:04:31.567 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:32.349 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:32.346+0000 7f46963c98c0 -1 osd.1 0 log_to_monitors true 2026-02-26T03:04:32.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:32.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:32.569 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:04:32.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:04:32.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:32.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:04:32.764 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:33.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:33.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:33.765 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:04:33.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:04:33.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:33.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:04:33.964 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2515308898,v1:127.0.0.1:6811/2515308898] [v2:127.0.0.1:6812/2515308898,v1:127.0.0.1:6813/2515308898] exists,up 06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 2 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:04:33.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:04:33.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:04:33.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:04:33.971 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:04:33.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6332a98b-2324-4f74-af7c-edacae136c3b 2026-02-26T03:04:33.972 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 6332a98b-2324-4f74-af7c-edacae136c3b 2026-02-26T03:04:33.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 6332a98b-2324-4f74-af7c-edacae136c3b' 2026-02-26T03:04:33.973 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:04:33.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBBuJ9pUhGjOhAAtx0JvCg38K1vs2w+0I8nvg== 2026-02-26T03:04:33.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBBuJ9pUhGjOhAAtx0JvCg38K1vs2w+0I8nvg=="}' 2026-02-26T03:04:33.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6332a98b-2324-4f74-af7c-edacae136c3b -i td/divergent-priors/2/new.json 2026-02-26T03:04:34.186 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:04:34.195 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-26T03:04:34.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBuJ9pUhGjOhAAtx0JvCg38K1vs2w+0I8nvg== --osd-uuid 6332a98b-2324-4f74-af7c-edacae136c3b 2026-02-26T03:04:34.214 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:34.214+0000 7f15cccb88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:34.215 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:34.214+0000 7f15cccb88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:34.216 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:34.214+0000 7f15cccb88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:34.217 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:34.214+0000 7f15cccb88c0 -1 bdev(0x55d26ab45c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-26T03:04:34.217 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:34.214+0000 7f15cccb88c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-26T03:04:36.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-26T03:04:36.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:04:36.513 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T03:04:36.513 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T03:04:36.513 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:04:36.788 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T03:04:36.788 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:04:36.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:04:36.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:04:36.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:04:36.790 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:04:36.817 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:36.814+0000 7f299d9718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:36.818 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:36.818+0000 7f299d9718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:36.819 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:36.818+0000 7f299d9718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:04:36.987 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T03:04:36.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:04:36.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:04:36.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:04:36.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:04:36.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:36.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:04:36.989 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:04:36.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:36.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:04:37.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:38.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:38.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:38.191 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:04:38.191 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:04:38.191 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:38.191 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:04:38.319 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:38.317+0000 7f299d9718c0 -1 Falling back to public interface 2026-02-26T03:04:38.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:39.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:39.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:39.393 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:04:39.394 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:04:39.394 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:39.395 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:04:39.589 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:40.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:40.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:40.591 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:04:40.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:04:40.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:40.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:04:40.787 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:41.535 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:04:41.533+0000 7f299d9718c0 -1 osd.2 0 log_to_monitors true 2026-02-26T03:04:41.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:41.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:41.789 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:04:41.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:04:41.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:41.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:04:41.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:04:42.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:04:42.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:04:42.994 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:04:42.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:04:42.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:04:42.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:04:43.197 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4004339502,v1:127.0.0.1:6819/4004339502] [v2:127.0.0.1:6820/4004339502,v1:127.0.0.1:6821/4004339502] exists,up 6332a98b-2324-4f74-af7c-edacae136c3b 2026-02-26T03:04:43.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:04:43.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:04:43.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:04:43.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:643: TEST_divergent_3: ceph osd set noout 2026-02-26T03:04:43.495 INFO:tasks.workunit.client.0.trial123.stderr:noout is set 2026-02-26T03:04:43.508 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:644: TEST_divergent_3: ceph osd set noin 2026-02-26T03:04:43.800 INFO:tasks.workunit.client.0.trial123.stderr:noin is set 2026-02-26T03:04:43.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:645: TEST_divergent_3: ceph osd set nodown 2026-02-26T03:04:44.106 INFO:tasks.workunit.client.0.trial123.stderr:nodown is set 2026-02-26T03:04:44.115 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:646: TEST_divergent_3: create_pool test 1 1 2026-02-26T03:04:44.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-26T03:04:44.356 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' already exists 2026-02-26T03:04:44.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T03:04:45.366 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:647: TEST_divergent_3: ceph osd pool set test size 3 2026-02-26T03:04:45.666 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 size to 3 2026-02-26T03:04:45.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:648: TEST_divergent_3: ceph osd pool set test min_size 2 2026-02-26T03:04:45.972 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 min_size to 2 2026-02-26T03:04:45.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:651: TEST_divergent_3: ceph osd pool set test pg_autoscale_mode on 2026-02-26T03:04:46.277 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 pg_autoscale_mode to on 2026-02-26T03:04:46.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:653: TEST_divergent_3: divergent=-1 2026-02-26T03:04:46.291 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: date +%s 2026-02-26T03:04:46.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: start_time=1772075086 2026-02-26T03:04:46.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:655: TEST_divergent_3: max_duration=300 2026-02-26T03:04:46.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: '[' -1 -le -1 ']' 2026-02-26T03:04:46.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: flush_pg_stats 2026-02-26T03:04:46.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:04:46.293 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:04:46.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:04:46.497 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:04:46.497 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:04:46.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:04:46.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:46.497 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:04:46.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-26T03:04:46.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-26T03:04:46.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-26T03:04:46.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:46.576 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:04:46.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-26T03:04:46.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-26T03:04:46.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-26T03:04:46.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:46.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:04:46.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-26T03:04:46.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-26T03:04:46.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-60129542147' 2026-02-26T03:04:46.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:46.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-26T03:04:46.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:46.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:04:46.742 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-26T03:04:46.743 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:46.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-26T03:04:46.744 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836487 2026-02-26T03:04:46.745 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-26T03:04:46.745 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:04:46.945 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-26T03:04:46.945 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:04:47.946 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:04:47.947 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:04:48.148 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-26T03:04:48.148 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:48.149 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-26T03:04:48.149 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:48.150 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:04:48.151 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-26T03:04:48.151 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:48.152 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-26T03:04:48.152 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672965 2026-02-26T03:04:48.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-26T03:04:48.153 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:04:48.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672965 -lt 42949672965 2026-02-26T03:04:48.366 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:48.366 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-26T03:04:48.366 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:48.367 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:04:48.368 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-26T03:04:48.368 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:48.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-26T03:04:48.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-26T03:04:48.370 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542147 2026-02-26T03:04:48.370 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:04:48.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-26T03:04:48.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:660: TEST_divergent_3: wait_for_clean 2026-02-26T03:04:48.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:04:48.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:04:48.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:04:48.575 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:04:48.576 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:04:48.576 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:04:48.576 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:04:48.576 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:04:48.576 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:04:48.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:04:48.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:04:48.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:04:48.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:04:48.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:04:48.679 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:04:48.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:04:48.882 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:04:48.882 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:04:48.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:04:48.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:48.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:04:48.960 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-26T03:04:48.960 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-26T03:04:48.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-26T03:04:48.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:48.961 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:04:49.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-26T03:04:49.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-26T03:04:49.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2026-02-26T03:04:49.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:49.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:04:49.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-26T03:04:49.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-26T03:04:49.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-60129542150' 2026-02-26T03:04:49.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:49.120 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-26T03:04:49.120 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:49.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:04:49.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-26T03:04:49.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:49.123 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-26T03:04:49.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-26T03:04:49.124 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836489 2026-02-26T03:04:49.124 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:04:49.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-26T03:04:49.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:04:50.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:04:50.331 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:04:50.524 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-26T03:04:50.524 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:50.525 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-26T03:04:50.526 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:50.526 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:04:50.527 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-26T03:04:50.527 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:50.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-26T03:04:50.529 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672968 2026-02-26T03:04:50.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-26T03:04:50.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:04:50.734 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-26T03:04:50.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:50.735 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-26T03:04:50.735 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:50.736 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:04:50.737 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-26T03:04:50.737 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:50.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-26T03:04:50.739 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542150 2026-02-26T03:04:50.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-26T03:04:50.740 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:04:50.941 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-26T03:04:50.941 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:04:50.942 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:04:50.942 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:04:51.215 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:04:51.215 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:04:51.216 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:04:51.216 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:04:51.216 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:04:51.216 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:04:51.216 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:04:51.216 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:04:51.417 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:04:51.417 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:04:51.417 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:04:51.418 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:04:51.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:04:51.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:04:51.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:04:51.683 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: ceph pg dump pgs --format=json 2026-02-26T03:04:51.683 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: jq '.pg_stats[0].up_primary' 2026-02-26T03:04:51.875 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:04:51.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: divergent=1 2026-02-26T03:04:51.885 INFO:tasks.workunit.client.0.trial123.stdout:primary and soon to be divergent is 1 2026-02-26T03:04:51.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:664: TEST_divergent_3: echo 'primary and soon to be divergent is 1' 2026-02-26T03:04:51.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:04:52.071 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:04:52.071 INFO:tasks.workunit.client.0.trial123.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-26T03:04:45.413359+0000 0'0 29:23 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:06:47.790931+0000 0 0 2026-02-26T03:04:52.072 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:04:52.072 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:04:52.072 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:04:52.079 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: date +%s 2026-02-26T03:04:52.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: current_time=1772075092 2026-02-26T03:04:52.081 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:668: TEST_divergent_3: expr 1772075092 - 1772075086 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:668: TEST_divergent_3: elapsed_time=6 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: '[' 6 -gt 300 ']' 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: '[' 1 -le -1 ']' 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:675: TEST_divergent_3: local non_divergent= 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 0 = 1 ']' 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: non_divergent=' 0' 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 1 = 1 ']' 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:679: TEST_divergent_3: continue 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-26T03:04:52.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 2 = 1 ']' 2026-02-26T03:04:52.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: non_divergent=' 0 2' 2026-02-26T03:04:52.083 INFO:tasks.workunit.client.0.trial123.stdout:writing initial objects 2026-02-26T03:04:52.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:684: TEST_divergent_3: echo 'writing initial objects' 2026-02-26T03:04:52.083 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: seq 1 100 2026-02-26T03:04:52.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_1 /etc/fstab 2026-02-26T03:04:52.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_2 /etc/fstab 2026-02-26T03:04:52.134 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.134 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_3 /etc/fstab 2026-02-26T03:04:52.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_4 /etc/fstab 2026-02-26T03:04:52.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_5 /etc/fstab 2026-02-26T03:04:52.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_6 /etc/fstab 2026-02-26T03:04:52.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_7 /etc/fstab 2026-02-26T03:04:52.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_8 /etc/fstab 2026-02-26T03:04:52.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_9 /etc/fstab 2026-02-26T03:04:52.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_10 /etc/fstab 2026-02-26T03:04:52.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.345 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_11 /etc/fstab 2026-02-26T03:04:52.371 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.371 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_12 /etc/fstab 2026-02-26T03:04:52.397 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.397 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_13 /etc/fstab 2026-02-26T03:04:52.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.427 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_14 /etc/fstab 2026-02-26T03:04:52.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_15 /etc/fstab 2026-02-26T03:04:52.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_16 /etc/fstab 2026-02-26T03:04:52.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_17 /etc/fstab 2026-02-26T03:04:52.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_18 /etc/fstab 2026-02-26T03:04:52.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_19 /etc/fstab 2026-02-26T03:04:52.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_20 /etc/fstab 2026-02-26T03:04:52.592 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.592 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_21 /etc/fstab 2026-02-26T03:04:52.617 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.617 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_22 /etc/fstab 2026-02-26T03:04:52.644 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.644 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_23 /etc/fstab 2026-02-26T03:04:52.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_24 /etc/fstab 2026-02-26T03:04:52.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_25 /etc/fstab 2026-02-26T03:04:52.714 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.714 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_26 /etc/fstab 2026-02-26T03:04:52.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_27 /etc/fstab 2026-02-26T03:04:52.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_28 /etc/fstab 2026-02-26T03:04:52.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_29 /etc/fstab 2026-02-26T03:04:52.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_30 /etc/fstab 2026-02-26T03:04:52.851 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.851 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_31 /etc/fstab 2026-02-26T03:04:52.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_32 /etc/fstab 2026-02-26T03:04:52.908 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.908 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_33 /etc/fstab 2026-02-26T03:04:52.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_34 /etc/fstab 2026-02-26T03:04:52.963 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.964 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_35 /etc/fstab 2026-02-26T03:04:52.991 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:52.991 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_36 /etc/fstab 2026-02-26T03:04:53.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_37 /etc/fstab 2026-02-26T03:04:53.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_38 /etc/fstab 2026-02-26T03:04:53.077 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.077 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_39 /etc/fstab 2026-02-26T03:04:53.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_40 /etc/fstab 2026-02-26T03:04:53.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_41 /etc/fstab 2026-02-26T03:04:53.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_42 /etc/fstab 2026-02-26T03:04:53.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_43 /etc/fstab 2026-02-26T03:04:53.192 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.192 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_44 /etc/fstab 2026-02-26T03:04:53.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_45 /etc/fstab 2026-02-26T03:04:53.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_46 /etc/fstab 2026-02-26T03:04:53.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_47 /etc/fstab 2026-02-26T03:04:53.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_48 /etc/fstab 2026-02-26T03:04:53.331 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_49 /etc/fstab 2026-02-26T03:04:53.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.352 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_50 /etc/fstab 2026-02-26T03:04:53.373 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.373 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_51 /etc/fstab 2026-02-26T03:04:53.401 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.401 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_52 /etc/fstab 2026-02-26T03:04:53.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_53 /etc/fstab 2026-02-26T03:04:53.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_54 /etc/fstab 2026-02-26T03:04:53.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_55 /etc/fstab 2026-02-26T03:04:53.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_56 /etc/fstab 2026-02-26T03:04:53.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_57 /etc/fstab 2026-02-26T03:04:53.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_58 /etc/fstab 2026-02-26T03:04:53.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.592 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_59 /etc/fstab 2026-02-26T03:04:53.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_60 /etc/fstab 2026-02-26T03:04:53.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_61 /etc/fstab 2026-02-26T03:04:53.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_62 /etc/fstab 2026-02-26T03:04:53.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_63 /etc/fstab 2026-02-26T03:04:53.724 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_64 /etc/fstab 2026-02-26T03:04:53.752 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.752 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_65 /etc/fstab 2026-02-26T03:04:53.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.780 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_66 /etc/fstab 2026-02-26T03:04:53.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_67 /etc/fstab 2026-02-26T03:04:53.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_68 /etc/fstab 2026-02-26T03:04:53.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_69 /etc/fstab 2026-02-26T03:04:53.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_70 /etc/fstab 2026-02-26T03:04:53.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.920 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_71 /etc/fstab 2026-02-26T03:04:53.948 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.948 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_72 /etc/fstab 2026-02-26T03:04:53.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:53.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_73 /etc/fstab 2026-02-26T03:04:54.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_74 /etc/fstab 2026-02-26T03:04:54.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_75 /etc/fstab 2026-02-26T03:04:54.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_76 /etc/fstab 2026-02-26T03:04:54.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_77 /etc/fstab 2026-02-26T03:04:54.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_78 /etc/fstab 2026-02-26T03:04:54.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_79 /etc/fstab 2026-02-26T03:04:54.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_80 /etc/fstab 2026-02-26T03:04:54.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_81 /etc/fstab 2026-02-26T03:04:54.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_82 /etc/fstab 2026-02-26T03:04:54.255 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.256 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_83 /etc/fstab 2026-02-26T03:04:54.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_84 /etc/fstab 2026-02-26T03:04:54.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_85 /etc/fstab 2026-02-26T03:04:54.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_86 /etc/fstab 2026-02-26T03:04:54.367 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_87 /etc/fstab 2026-02-26T03:04:54.396 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.396 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_88 /etc/fstab 2026-02-26T03:04:54.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_89 /etc/fstab 2026-02-26T03:04:54.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_90 /etc/fstab 2026-02-26T03:04:54.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_91 /etc/fstab 2026-02-26T03:04:54.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_92 /etc/fstab 2026-02-26T03:04:54.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_93 /etc/fstab 2026-02-26T03:04:54.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.563 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_94 /etc/fstab 2026-02-26T03:04:54.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_95 /etc/fstab 2026-02-26T03:04:54.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_96 /etc/fstab 2026-02-26T03:04:54.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_97 /etc/fstab 2026-02-26T03:04:54.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_98 /etc/fstab 2026-02-26T03:04:54.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_99 /etc/fstab 2026-02-26T03:04:54.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-26T03:04:54.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_100 /etc/fstab 2026-02-26T03:04:54.748 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:691: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:04:54.749 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:691: TEST_divergent_3: wait_for_clean 2026-02-26T03:04:54.749 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:04:54.749 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:04:54.749 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:04:54.749 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:04:54.750 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:04:54.750 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:04:54.750 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:04:54.750 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:04:54.750 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:04:54.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:04:54.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:04:54.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:04:54.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:04:54.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:04:54.822 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:04:55.021 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:04:55.021 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:04:55.021 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:04:55.021 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:04:55.021 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:55.022 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:04:55.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-26T03:04:55.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-26T03:04:55.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-26T03:04:55.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:55.103 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:04:55.181 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672971 2026-02-26T03:04:55.181 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672971 2026-02-26T03:04:55.181 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-42949672971' 2026-02-26T03:04:55.181 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:04:55.181 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:04:55.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-26T03:04:55.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-26T03:04:55.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-42949672971 2-60129542153' 2026-02-26T03:04:55.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:55.265 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-26T03:04:55.265 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:55.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:04:55.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-26T03:04:55.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:55.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-26T03:04:55.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-26T03:04:55.268 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836493 2026-02-26T03:04:55.268 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:04:55.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-26T03:04:55.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:55.477 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672971 2026-02-26T03:04:55.477 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:55.478 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:04:55.479 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672971 2026-02-26T03:04:55.479 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:55.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672971 2026-02-26T03:04:55.481 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672971 2026-02-26T03:04:55.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-26T03:04:55.482 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:04:55.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-26T03:04:55.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:04:55.696 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-26T03:04:55.696 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:04:55.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:04:55.698 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-26T03:04:55.698 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:04:55.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-26T03:04:55.700 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-26T03:04:55.700 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542153 2026-02-26T03:04:55.700 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:04:55.908 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-26T03:04:55.909 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:04:55.909 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:04:55.909 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:04:56.182 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:04:56.380 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:04:56.381 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:04:56.381 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:04:56.381 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:04:56.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:04:56.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:04:56.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:04:56.640 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: get_pg test existing_1 2026-02-26T03:04:56.640 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-26T03:04:56.640 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-26T03:04:56.640 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-26T03:04:56.640 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-26T03:04:56.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: local pgid=1.0 2026-02-26T03:04:56.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:696: TEST_divergent_3: echo 'blackholing osds 0 2' 2026-02-26T03:04:56.837 INFO:tasks.workunit.client.0.trial123.stdout:blackholing osds 0 2 2026-02-26T03:04:56.838 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:697: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:04:57.030 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:04:57.030 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-26T03:04:45.413359+0000 29'100 29:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:06:47.790931+0000 0 0 2026-02-26T03:04:57.030 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:04:57.030 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:04:57.031 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:04:57.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:04:57.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: get_asok_path osd.0 2026-02-26T03:04:57.041 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:04:57.041 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:04:57.041 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:04:57.041 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:57.041 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:57.041 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:04:57.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: CEPH_ARGS= 2026-02-26T03:04:57.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-26T03:04:57.094 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:04:57.094 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:04:57.094 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:04:57.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: get_asok_path osd.2 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:04:57.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: CEPH_ARGS= 2026-02-26T03:04:57.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-26T03:04:57.151 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:04:57.151 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:04:57.152 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:04:57.157 INFO:tasks.workunit.client.0.trial123.stdout:create new divergent objects 2026-02-26T03:04:57.157 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: echo 'create new divergent objects' 2026-02-26T03:04:57.158 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: seq 1 2 2026-02-26T03:04:57.158 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:04:57.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:04:57.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: rados -p test create newobject_1 2026-02-26T03:04:57.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:710: TEST_divergent_3: echo 'writing divergent objects' 2026-02-26T03:04:57.159 INFO:tasks.workunit.client.0.trial123.stdout:writing divergent objects 2026-02-26T03:04:57.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: rados -p test create newobject_2 2026-02-26T03:04:57.160 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: seq 1 2 2026-02-26T03:04:57.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-26T03:04:57.161 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-26T03:04:57.161 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: rados -p test put existing_1 /etc/resolv.conf 2026-02-26T03:04:57.161 INFO:tasks.workunit.client.0.trial123.stdout:remove divergent objects 2026-02-26T03:04:57.162 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:716: TEST_divergent_3: echo 'remove divergent objects' 2026-02-26T03:04:57.162 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: rados -p test put existing_2 /etc/resolv.conf 2026-02-26T03:04:57.162 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: seq 1 2 2026-02-26T03:04:57.163 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-26T03:04:57.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: expr 1 + 2 2026-02-26T03:04:57.163 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: rmi=3 2026-02-26T03:04:57.164 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-26T03:04:57.164 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: rados -p test rm existing_3 2026-02-26T03:04:57.164 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: expr 2 + 2 2026-02-26T03:04:57.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: rmi=4 2026-02-26T03:04:57.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: sleep 10 2026-02-26T03:04:57.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: rados -p test rm existing_4 2026-02-26T03:05:07.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:723: TEST_divergent_3: killall -9 rados 2026-02-26T03:05:07.208 INFO:tasks.workunit.client.0.trial123.stderr:rados: no process found 2026-02-26T03:05:07.208 INFO:tasks.workunit.client.0.trial123.stdout:killing all the osds 2026-02-26T03:05:07.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:726: TEST_divergent_3: echo 'killing all the osds' 2026-02-26T03:05:07.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:727: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:05:07.407 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:05:07.407 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 active+clean 2026-02-26T03:04:45.413359+0000 29'106 29:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:06:47.790931+0000 0 0 2026-02-26T03:05:07.407 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:05:07.407 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:05:07.407 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:05:07.417 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd 2026-02-26T03:05:07.419 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:05:07.419 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:05:07.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:05:07.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:05:07.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:05:07.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:05:07.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-26T03:05:07.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.0 2026-02-26T03:05:07.775 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:05:07.784 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-26T03:05:07.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.1 2026-02-26T03:05:08.044 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:05:08.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-26T03:05:08.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.2 2026-02-26T03:05:08.352 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:05:08.362 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:05:08.362 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: ceph osd out osd.0 2026-02-26T03:05:08.626 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already out. 2026-02-26T03:05:08.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:05:08.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: ceph osd out osd.2 2026-02-26T03:05:08.891 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already out. 2026-02-26T03:05:08.901 INFO:tasks.workunit.client.0.trial123.stdout:bringing up non_divergent 0 2 2026-02-26T03:05:08.901 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:739: TEST_divergent_3: echo 'bringing up non_divergent 0 2' 2026-02-26T03:05:08.901 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:740: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:05:09.098 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:05:09.098 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 stale+active+clean 2026-02-26T03:04:45.413359+0000 29'106 29:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:06:47.790931+0000 0 0 2026-02-26T03:05:09.098 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:05:09.099 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:05:09.099 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: activate_osd td/divergent-priors 0 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:05:09.108 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:05:09.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:05:09.110 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:05:09.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:05:09.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:05:09.111 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-26T03:05:09.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:05:09.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:05:09.113 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:05:09.113 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:05:09.136 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:09.132+0000 7f9d526ed8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:05:09.137 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:09.132+0000 7f9d526ed8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:05:09.138 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:09.132+0000 7f9d526ed8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:05:09.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:05:09.319 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:05:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:05:09.524 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:10.525 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:10.525 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:10.525 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:05:10.525 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:05:10.525 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:10.525 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:05:10.621 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:10.616+0000 7f9d526ed8c0 -1 Falling back to public interface 2026-02-26T03:05:10.729 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:11.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:11.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:11.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:05:11.731 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:05:11.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:11.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:05:11.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:12.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:12.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:12.932 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:05:12.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:05:12.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:12.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:05:13.127 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:13.635 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:13.628+0000 7f9d526ed8c0 -1 osd.0 29 log_to_monitors true 2026-02-26T03:05:14.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:14.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:14.129 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:05:14.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:05:14.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:14.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:05:14.328 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up out weight 0 up_from 36 up_thru 0 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/2144727426,v1:127.0.0.1:6803/2144727426] [v2:127.0.0.1:6804/2144727426,v1:127.0.0.1:6805/2144727426] exists,up 03f68cfd-282b-40c9-8b38-71cb31fbe97b 2026-02-26T03:05:14.331 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:05:14.331 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:05:14.331 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: activate_osd td/divergent-priors 2 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:05:14.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:05:14.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:05:14.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:05:14.335 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:05:14.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:05:14.353 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:14.348+0000 7fb7a80e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:05:14.355 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:14.348+0000 7fb7a80e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:05:14.356 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:14.352+0000 7fb7a80e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:05:14.535 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:05:14.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:14.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:05:14.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:15.641 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:15.635+0000 7fb7a80e48c0 -1 Falling back to public interface 2026-02-26T03:05:15.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:15.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:15.741 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:05:15.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:05:15.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:15.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:05:15.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:16.940 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:16.940 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:16.941 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:05:16.941 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:05:16.941 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:16.941 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:05:17.141 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:18.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:18.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:18.143 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:05:18.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:05:18.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:18.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:05:18.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:05:18.702 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:05:18.699+0000 7fb7a80e48c0 -1 osd.2 29 log_to_monitors true 2026-02-26T03:05:19.341 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:05:19.341 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:05:19.341 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:05:19.341 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:05:19.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:05:19.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:05:19.543 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up out weight 0 up_from 39 up_thru 0 down_at 32 last_clean_interval [14,29) [v2:127.0.0.1:6810/3386818234,v1:127.0.0.1:6811/3386818234] [v2:127.0.0.1:6812/3386818234,v1:127.0.0.1:6813/3386818234] exists,up 6332a98b-2324-4f74-af7c-edacae136c3b 2026-02-26T03:05:19.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:05:19.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:05:19.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:05:19.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:05:19.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph osd in osd.0 2026-02-26T03:05:19.790 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already in. 2026-02-26T03:05:19.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:05:19.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph osd in osd.2 2026-02-26T03:05:20.095 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already in. 2026-02-26T03:05:20.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:05:20.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: wait_for_clean 2026-02-26T03:05:20.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:05:20.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:05:20.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:05:20.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:05:20.106 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:05:20.106 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:05:20.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:05:20.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:05:20.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:05:20.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:05:20.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:05:20.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:05:20.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:05:20.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:05:20.174 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:05:20.373 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:05:20.373 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:05:20.373 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:05:20.373 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:05:20.373 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:05:20.373 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:05:20.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822660 2026-02-26T03:05:20.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822660 2026-02-26T03:05:20.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822660' 2026-02-26T03:05:20.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:05:20.454 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:05:20.513 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:05:20.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:05:20.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:05:20.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:05:20.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:05:20.517 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:05:20.593 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724547 2026-02-26T03:05:20.593 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724547 2026-02-26T03:05:20.593 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822660 2-167503724547' 2026-02-26T03:05:20.594 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:05:20.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-154618822660 2026-02-26T03:05:20.594 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:05:20.596 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:05:20.597 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-154618822660 2026-02-26T03:05:20.597 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:05:20.598 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822660 2026-02-26T03:05:20.598 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 154618822660' 2026-02-26T03:05:20.598 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 154618822660 2026-02-26T03:05:20.599 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:05:20.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822657 -lt 154618822660 2026-02-26T03:05:20.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:05:21.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:05:21.821 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:05:22.018 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-26T03:05:22.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:05:22.019 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-167503724547 2026-02-26T03:05:22.019 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:05:22.021 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:05:22.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-167503724547 2026-02-26T03:05:22.022 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:05:22.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724547 2026-02-26T03:05:22.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 167503724547' 2026-02-26T03:05:22.023 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 167503724547 2026-02-26T03:05:22.024 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:05:22.233 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-26T03:05:22.234 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:05:22.234 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:22.235 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:22.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 == 0 2026-02-26T03:05:22.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:22.501 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:22.501 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:22.501 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:22.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:22.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:22.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:22.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:22.709 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:22.710 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:22.710 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:22.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:22.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:05:22.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:05:22.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:05:22.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:22.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:05:23.075 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:23.076 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:23.076 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:23.077 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:23.077 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:23.077 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:23.077 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:23.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:23.279 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:23.279 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:23.280 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:23.280 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:23.542 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:23.543 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:23.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:23.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:23.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:05:23.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:23.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:05:24.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:24.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:24.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:24.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:24.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:24.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:24.009 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:24.009 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:24.213 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:24.214 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:24.214 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:24.214 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:24.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:24.473 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:24.473 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:24.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:24.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:24.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:05:24.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:24.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:05:25.139 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:25.139 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:25.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:25.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:25.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:25.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:25.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:25.140 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:25.345 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:25.345 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:25.345 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:25.345 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:25.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:25.610 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:25.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:25.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:25.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:05:25.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:25.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:05:26.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:26.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:26.665 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:26.665 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:26.665 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:26.665 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:26.666 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:26.666 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:26.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:26.865 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:26.866 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:26.866 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:27.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:27.130 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:27.130 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:27.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:27.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:27.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:05:27.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:27.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:05:28.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:28.993 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:28.993 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:28.993 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:28.993 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:28.993 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:28.994 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:28.994 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:29.195 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:29.196 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:29.196 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:29.196 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:29.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:29.455 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:29.455 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:29.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:29.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:29.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:05:29.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:29.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:05:32.921 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:32.922 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:32.922 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:32.922 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:32.922 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:32.922 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:32.923 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:32.923 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:33.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:33.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:33.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:33.122 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:33.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:33.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:33.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:33.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:33.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:33.386 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:33.386 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:33.386 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:33.386 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:33.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:33.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:33.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:05:33.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:33.638 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:05:40.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:40.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:40.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:40.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:40.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:40.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:40.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:40.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:40.243 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:40.243 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:40.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:40.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:40.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:40.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:40.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:40.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:40.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:40.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:40.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:40.512 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:40.512 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:40.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:40.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:40.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:05:40.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:40.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:05:48.076 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:48.076 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:48.077 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:48.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:48.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:48.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:48.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:48.078 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:48.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:48.286 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:48.286 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:48.287 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:48.556 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:48.557 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:48.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:48.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:48.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:05:48.824 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:05:49.076 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "44cb29f9-3663-4694-9a55-5c7e3128ab87", 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:05:49.075265+0000", 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:05:49.077 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:05:49.078 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "count": 32 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:05:49.079 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.082 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.084 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.085 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-26T03:05:49.086 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.087 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:05:49.088 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:04:14.862220Z", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:04:14.862220Z", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:05:49.089 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.090 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:05:49.091 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 45, 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:04:14.949126+0000", 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:05:20.145638+0000", 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:05:18.857740+0000", 2026-02-26T03:05:49.092 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:05:20.043689+0000", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 9, 2026-02-26T03:05:49.093 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:05:49.094 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T03:04:44.268505+0000", 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 1, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool", 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 0, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:05:49.095 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "on", 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 32, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 32, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 32, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 32, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:05:49.096 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "40", 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "40", 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "selfmanaged", 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 0, 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 0, 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [], 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:05:49.097 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "", 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:05:49.098 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {}, 2026-02-26T03:05:49.099 INFO:tasks.workunit.client.0.trial123.stdout: "read_balance": { 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "score_type": "Fair distribution", 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "score_acting": 2.9900000095367432, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "score_stable": 2.9900000095367432, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_score": 0.67000001668930054, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_acting": 2, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_stable": 2, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity_weighted": 1, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity": 1, 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity_weighted": 1 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:05:49.100 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "03f68cfd-282b-40c9-8b38-71cb31fbe97b", 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 5, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 29, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 36, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 44, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:05:49.101 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.102 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:05:49.103 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:05:49.104 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/2144727426", 2026-02-26T03:05:49.105 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/2144727426", 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/2144727426", 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/2144727426", 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "06feee01-2860-490b-b9ae-8a404b84c218", 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:05:49.106 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 10, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 21, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 31, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.107 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.108 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.109 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.110 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/2515308898", 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/2515308898", 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2515308898", 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2515308898", 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "6332a98b-2324-4f74-af7c-edacae136c3b", 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:05:49.111 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 14, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 29, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 39, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 32, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:05:49.112 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.113 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.114 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3386818234", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3386818234", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3386818234", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3386818234", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.115 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:05:07.724821+0000", 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:04:23.955382+0000", 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:05:07.994012+0000", 2026-02-26T03:05:49.116 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:05:08.297504+0000", 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:05:49.117 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:04:42.555499+0000", 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:05:49.118 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.119 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/2144727426,v1:127.0.0.1:6805/2144727426]", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:05:49.120 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:04:17.564015Z", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/2144727426,v1:127.0.0.1:6803/2144727426]", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/2144727426,v1:127.0.0.1:6809/2144727426]", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/2144727426,v1:127.0.0.1:6807/2144727426]", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:05:49.121 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/2515308898,v1:127.0.0.1:6813/2515308898]", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:05:49.122 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:04:26.762573Z", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/2515308898,v1:127.0.0.1:6811/2515308898]", 2026-02-26T03:05:49.123 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2515308898,v1:127.0.0.1:6817/2515308898]", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2515308898,v1:127.0.0.1:6815/2515308898]", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3386818234,v1:127.0.0.1:6813/3386818234]", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:05:49.124 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:04:35.972451Z", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:05:49.125 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3386818234,v1:127.0.0.1:6811/3386818234]", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3386818234,v1:127.0.0.1:6817/3386818234]", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3386818234,v1:127.0.0.1:6815/3386818234]", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.126 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 0, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [] 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 36 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 40 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 45, 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:05:49.127 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.128 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.129 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stderr:report 3552007735 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:05:49.130 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:05:49.131 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:05:49.132 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:05:49.133 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:05:49.134 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 0, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.135 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T03:04:14:948935+0000", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.136 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 32, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 32, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 72800, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:05:49.137 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 100, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 100, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:05:49.138 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 512, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 512, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "up": 64, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 64, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 65, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 84004, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2168, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488796, 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:05:49.139 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036527104, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2220032, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 527268, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:05:49.140 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 65, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 84004, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2168, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488796, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036527104, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2220032, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 527268, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:05:49.141 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 32, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 72800, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 100, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 100, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:05:49.142 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1220608, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 217422, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:05:49.143 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 512, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 512, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "up": 64, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 64, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 154618822665 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 42949672973 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 167503724552 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "num": 32 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:05:49.144 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 32, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 32, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 32, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 4, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:05:35.420731+0000", 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "0": { 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:05:49.145 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 44, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 242, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 341, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:05:49.146 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:05:49.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:05:49.147 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:753: TEST_divergent_3: expr 2 + 2 2026-02-26T03:05:49.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:753: TEST_divergent_3: objname=existing_4 2026-02-26T03:05:49.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:754: TEST_divergent_3: echo 'writing non-divergent object existing_4' 2026-02-26T03:05:49.147 INFO:tasks.workunit.client.0.trial123.stdout:writing non-divergent object existing_4 2026-02-26T03:05:49.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:05:49.278 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:05:49.278 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.279 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:05:49.280 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:05:49.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:756: TEST_divergent_3: rados -p test put existing_4 /etc/resolv.conf 2026-02-26T03:05:49.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:05:49.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: wait_for_clean 2026-02-26T03:05:49.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:05:49.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:05:49.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:05:49.319 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:05:49.319 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:05:49.319 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:05:49.319 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:05:49.319 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:05:49.319 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:05:49.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:05:49.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:05:49.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:05:49.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:05:49.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:05:49.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:05:49.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:05:49.580 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:05:49.580 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:05:49.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:05:49.580 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:05:49.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:05:49.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822667 2026-02-26T03:05:49.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822667 2026-02-26T03:05:49.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822667' 2026-02-26T03:05:49.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:05:49.665 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:05:49.722 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:05:49.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:05:49.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:05:49.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:05:49.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:05:49.725 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:05:49.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724555 2026-02-26T03:05:49.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724555 2026-02-26T03:05:49.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822667 2-167503724555' 2026-02-26T03:05:49.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:05:49.803 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-154618822667 2026-02-26T03:05:49.803 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:05:49.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:05:49.805 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-154618822667 2026-02-26T03:05:49.805 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:05:49.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822667 2026-02-26T03:05:49.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 154618822667' 2026-02-26T03:05:49.807 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 154618822667 2026-02-26T03:05:49.807 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:05:50.031 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822665 -lt 154618822667 2026-02-26T03:05:50.031 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:05:51.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:05:51.032 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:05:51.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822665 -lt 154618822667 2026-02-26T03:05:51.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:05:52.230 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:05:52.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:05:52.438 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822668 -lt 154618822667 2026-02-26T03:05:52.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:05:52.439 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-167503724555 2026-02-26T03:05:52.440 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:05:52.440 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:05:52.441 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-167503724555 2026-02-26T03:05:52.442 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:05:52.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724555 2026-02-26T03:05:52.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 167503724555' 2026-02-26T03:05:52.444 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 167503724555 2026-02-26T03:05:52.444 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:05:52.649 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724555 -lt 167503724555 2026-02-26T03:05:52.650 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:05:52.650 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:52.650 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:52.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 == 0 2026-02-26T03:05:52.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:52.916 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:52.916 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:52.916 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:52.916 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:52.917 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:52.917 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:53.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:53.119 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:53.119 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:53.120 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:53.384 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:53.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:05:53.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:05:53.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:05:53.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:53.385 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:05:53.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:53.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:53.487 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:53.487 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:53.487 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:53.487 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:53.487 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:53.487 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:53.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:53.689 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:53.690 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:53.690 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:53.953 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:53.954 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:54.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:54.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:54.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:05:54.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:54.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:05:54.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:54.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:54.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:54.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:54.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:54.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:54.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:54.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:54.623 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:54.623 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:54.623 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:54.623 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:54.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:54.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:54.880 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:55.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:55.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:55.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:05:55.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:55.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:05:55.533 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:55.533 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:55.534 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:55.534 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:55.535 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:55.535 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:55.535 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:55.535 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:55.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:55.731 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:55.731 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:55.731 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:55.988 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:55.989 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:56.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:56.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:56.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:05:56.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:56.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:05:57.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:57.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:57.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:57.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:57.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:57.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:57.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:57.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:57.246 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:57.247 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:57.247 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:57.247 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:57.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:57.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:57.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:57.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:57.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:57.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:57.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:57.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:57.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:05:57.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:05:57.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:05:57.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:05:57.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:05:57.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:05:59.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:05:59.360 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:05:59.360 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:05:59.360 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:05:59.360 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:05:59.360 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:05:59.361 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:05:59.361 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:05:59.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:05:59.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:05:59.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:05:59.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:05:59.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:05:59.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:05:59.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:05:59.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:05:59.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:05:59.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:05:59.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:05:59.832 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:05:59.832 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:06:00.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:06:00.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:06:00.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:06:00.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:06:00.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:06:03.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:06:03.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:06:03.297 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:06:03.297 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:06:03.297 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:06:03.297 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:06:03.298 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:06:03.298 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:06:03.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:06:03.505 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:06:03.505 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:06:03.505 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:06:03.785 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:06:03.786 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:06:04.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:06:04.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:06:04.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:06:04.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:06:04.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:06:10.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:06:10.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:06:10.455 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:06:10.455 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:06:10.455 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:06:10.456 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:06:10.456 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:06:10.456 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:06:10.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:06:10.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:06:10.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:06:10.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:06:10.918 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:06:10.918 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:06:10.918 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:06:10.918 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:06:10.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:06:10.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:06:10.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:06:10.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:06:10.919 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:06:11.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:06:11.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:06:11.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:06:11.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:06:11.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:06:18.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:06:18.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:06:18.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:06:18.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:06:18.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:06:18.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:06:18.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:06:18.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:06:18.672 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:06:18.672 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:06:18.673 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:06:18.673 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:06:18.931 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:06:18.932 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:06:19.185 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:06:19.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:06:19.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:06:19.186 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "44cb29f9-3663-4694-9a55-5c7e3128ab87", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:06:19.448404+0000", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:06:19.450 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "count": 32 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.451 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.452 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.453 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:04:14.862220Z", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:04:14.862220Z", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:06:19.454 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 45, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:04:14.949126+0000", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:05:20.145638+0000", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:05:18.857740+0000", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:05:20.043689+0000", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:06:19.455 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 9, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T03:04:44.268505+0000", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 1, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 0, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "on", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 32, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 32, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 32, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 32, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:06:19.456 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "40", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "40", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "selfmanaged", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [], 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "", 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:06:19.457 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 0, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {}, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "read_balance": { 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "score_type": "Fair distribution", 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "score_acting": 2.9900000095367432, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "score_stable": 2.9900000095367432, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_score": 0.67000001668930054, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_acting": 2, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "raw_score_stable": 2, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity_weighted": 1, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity": 1, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "average_primary_affinity_weighted": 1 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "03f68cfd-282b-40c9-8b38-71cb31fbe97b", 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 5, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 29, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 36, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 44, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.458 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:06:19.459 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2144727426 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/2144727426", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/2144727426", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/2144727426", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/2144727426", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "06feee01-2860-490b-b9ae-8a404b84c218", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 10, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 21, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 31, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:06:19.460 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.461 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 2515308898 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/2515308898", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/2515308898", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2515308898", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2515308898", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "6332a98b-2324-4f74-af7c-edacae136c3b", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 14, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 29, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 39, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 32, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:06:19.462 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3386818234 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.463 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3386818234", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3386818234", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3386818234", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3386818234", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:05:07.724821+0000", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:04:23.955382+0000", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:05:07.994012+0000", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:05:08.297504+0000", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:04:42.555499+0000", 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:06:19.464 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/2144727426,v1:127.0.0.1:6805/2144727426]", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:06:19.465 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:04:17.564015Z", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/2144727426,v1:127.0.0.1:6803/2144727426]", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/2144727426,v1:127.0.0.1:6809/2144727426]", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/2144727426,v1:127.0.0.1:6807/2144727426]", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:06:19.466 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/2515308898,v1:127.0.0.1:6813/2515308898]", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:04:26.762573Z", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/2515308898,v1:127.0.0.1:6811/2515308898]", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:06:19.467 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2515308898,v1:127.0.0.1:6817/2515308898]", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2515308898,v1:127.0.0.1:6815/2515308898]", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3386818234,v1:127.0.0.1:6813/3386818234]", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:06:19.468 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:04:35.972451Z", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3386818234,v1:127.0.0.1:6811/3386818234]", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3386818234,v1:127.0.0.1:6817/3386818234]", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3386818234,v1:127.0.0.1:6815/3386818234]", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 0, 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [] 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 36 2026-02-26T03:06:19.469 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 40 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 45, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:06:19.470 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stderr:report 1490867229 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:06:19.471 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:06:19.472 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:06:19.473 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.474 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 0, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T03:04:14:948935+0000", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:06:19.475 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 32, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 32, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73039, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:06:19.476 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.477 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 513, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 513, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "up": 64, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 64, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 65, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 84004, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2168, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488796, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036527104, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2220032, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 527746, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:06:19.478 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 65, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 84004, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2168, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488796, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036527104, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2220032, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 527746, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:06:19.479 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 32, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73039, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 100, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 300, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 100, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 100, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:06:19.480 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1220608, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 217900, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 513, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 513, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "up": 64, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 64, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 154618822673 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.481 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 42949672973 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 167503724560 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num": 32 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 32, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 32, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 32, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:06:19.482 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 5, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:06:15.427344+0000", 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 59, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 301, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 415, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:06:19.483 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery 2026-02-26T03:06:19.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:761: TEST_divergent_3: echo 'delay recovery' 2026-02-26T03:06:19.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:762: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.648 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:05:52.561290+0000 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:19.649 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:19.650 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:19.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:06:19.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: get_asok_path osd.0 2026-02-26T03:06:19.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:06:19.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:06:19.659 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:06:19.659 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:06:19.659 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:06:19.659 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:06:19.659 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: CEPH_ARGS= 2026-02-26T03:06:19.660 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok set_recovery_delay 100000 2026-02-26T03:06:19.726 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:06:19.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: for i in $non_divergent 2026-02-26T03:06:19.732 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: get_asok_path osd.2 2026-02-26T03:06:19.732 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:06:19.732 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:06:19.733 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:06:19.733 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:06:19.733 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:06:19.733 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:06:19.733 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: CEPH_ARGS= 2026-02-26T03:06:19.733 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok set_recovery_delay 100000 2026-02-26T03:06:19.786 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:06:19.792 INFO:tasks.workunit.client.0.trial123.stdout:revive divergent 1 2026-02-26T03:06:19.792 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:769: TEST_divergent_3: echo 'revive divergent 1' 2026-02-26T03:06:19.792 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:770: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.984 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.985 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:05:52.561290+0000 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.986 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.987 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.987 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.987 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:19.987 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:19.987 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:19.987 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:19.992 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:771: TEST_divergent_3: ceph osd set noup 2026-02-26T03:06:20.295 INFO:tasks.workunit.client.0.trial123.stderr:noup is set 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:772: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:06:20.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:06:20.312 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:06:20.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:06:20.314 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:06:20.314 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:06:20.314 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:06:20.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:06:20.315 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:06:20.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:06:20.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:06:20.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:06:20.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:06:20.329 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:20.325+0000 7effbfcd88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:06:20.331 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:20.325+0000 7effbfcd88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:06:20.332 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:20.325+0000 7effbfcd88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:06:20.511 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-26T03:06:20.512 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:773: TEST_divergent_3: sleep 5 2026-02-26T03:06:21.554 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:21.549+0000 7effbfcd88c0 -1 Falling back to public interface 2026-02-26T03:06:24.575 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:24.568+0000 7effbfcd88c0 -1 osd.1 29 log_to_monitors true 2026-02-26T03:06:25.513 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery divergent 2026-02-26T03:06:25.513 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:775: TEST_divergent_3: echo 'delay recovery divergent' 2026-02-26T03:06:25.513 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:776: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660711+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660665+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660580+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660639+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660592+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660685+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660717+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660742+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660723+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660737+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660659+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660671+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660698+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660730+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660646+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660599+0000 0 0 2026-02-26T03:06:25.698 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660558+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660678+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 48:147 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660749+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660755+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660692+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660548+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660573+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660566+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660586+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660652+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660626+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660704+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660613+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660620+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660632+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660605+0000 0 0 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:25.699 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:25.707 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: get_asok_path osd.1 2026-02-26T03:06:25.707 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:06:25.707 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:06:25.707 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:06:25.707 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:06:25.707 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:06:25.708 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:06:25.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: CEPH_ARGS= 2026-02-26T03:06:25.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok set_recovery_delay 100000 2026-02-26T03:06:25.762 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:06:25.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: ceph osd unset noup 2026-02-26T03:06:26.083 INFO:tasks.workunit.client.0.trial123.stderr:noup is unset 2026-02-26T03:06:26.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:781: TEST_divergent_3: wait_for_osd up 0 2026-02-26T03:06:26.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:06:26.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:06:26.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:06:26.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:06:26.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:06:26.098 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:06:26.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:06:26.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:06:26.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:06:26.302 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 36 up_thru 44 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/2144727426,v1:127.0.0.1:6803/2144727426] [v2:127.0.0.1:6804/2144727426,v1:127.0.0.1:6805/2144727426] exists,up 03f68cfd-282b-40c9-8b38-71cb31fbe97b 2026-02-26T03:06:26.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:782: TEST_divergent_3: wait_for_osd up 1 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:06:26.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:06:26.305 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:06:26.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:06:26.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:06:26.497 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/3367337205,v1:127.0.0.1:6819/3367337205] [v2:127.0.0.1:6820/3367337205,v1:127.0.0.1:6821/3367337205] exists,up 06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:06:26.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:783: TEST_divergent_3: wait_for_osd up 2 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:06:26.499 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:06:26.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:06:26.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:06:26.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:06:26.695 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 39 up_thru 0 down_at 32 last_clean_interval [14,29) [v2:127.0.0.1:6810/3386818234,v1:127.0.0.1:6811/3386818234] [v2:127.0.0.1:6812/3386818234,v1:127.0.0.1:6813/3386818234] exists,up 6332a98b-2324-4f74-af7c-edacae136c3b 2026-02-26T03:06:26.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:06:26.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:06:26.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:06:26.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:785: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:26.883 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:26.883 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660711+0000 0 0 2026-02-26T03:06:26.883 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660665+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660580+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660639+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660592+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660685+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660717+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660742+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660723+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660737+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660659+0000 0 0 2026-02-26T03:06:26.884 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660671+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660698+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660730+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660646+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660599+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660558+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660678+0000 0 0 2026-02-26T03:06:26.885 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 48:147 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660749+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660755+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660692+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660548+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660573+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660566+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660586+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660652+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660626+0000 0 0 2026-02-26T03:06:26.886 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660704+0000 0 0 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660613+0000 0 0 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660620+0000 0 0 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660632+0000 0 0 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660605+0000 0 0 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:26.887 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:26.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:786: TEST_divergent_3: echo 'wait for peering' 2026-02-26T03:06:26.894 INFO:tasks.workunit.client.0.trial123.stdout:wait for peering 2026-02-26T03:06:26.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:787: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660711+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660665+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660580+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660639+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660592+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660685+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660717+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660742+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660723+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660737+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660659+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660671+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660698+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660730+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660646+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660599+0000 0 0 2026-02-26T03:06:27.081 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660558+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660678+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 48:147 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660749+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660755+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660692+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660548+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660573+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660566+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660586+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660652+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660626+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660704+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660613+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660620+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660632+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660605+0000 0 0 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:27.082 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:27.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:788: TEST_divergent_3: rados -p test put foo /etc/fstab 2026-02-26T03:06:27.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: echo 'killing divergent 1' 2026-02-26T03:06:27.121 INFO:tasks.workunit.client.0.trial123.stdout:killing divergent 1 2026-02-26T03:06:27.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:793: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:27.314 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:27.314 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660711+0000 0 0 2026-02-26T03:06:27.314 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660665+0000 0 0 2026-02-26T03:06:27.314 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660580+0000 0 0 2026-02-26T03:06:27.314 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660639+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660592+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660685+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660717+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660742+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660723+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660737+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660659+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660671+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660698+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660730+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660646+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660599+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660558+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660678+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 48:147 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660749+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660755+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660692+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660548+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660573+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660566+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660586+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660652+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660626+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660704+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660613+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660620+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660632+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660605+0000 0 0 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:27.315 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:27.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:794: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd.1 2026-02-26T03:06:27.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:06:27.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:06:27.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:06:27.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:06:27.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: expfile=td/divergent-priors/exp.18886.out 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:798: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.18886.out 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-26T03:06:27.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.18886.out 2026-02-26T03:06:28.112 INFO:tasks.workunit.client.0.trial123.stderr:Exporting 1.0 info 1.0( v 29'100 (29'90,29'100] local-lis/les=50/51 n=3 ec=21/21 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-26T03:06:28.113 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-26T03:06:28.113 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-26T03:06:28.114 INFO:tasks.workunit.client.0.trial123.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-26T03:06:28.114 INFO:tasks.workunit.client.0.trial123.stderr:Export successful 2026-02-26T03:06:28.114 INFO:tasks.workunit.client.0.trial123.stdout: marking collection for removal 2026-02-26T03:06:28.114 INFO:tasks.workunit.client.0.trial123.stdout:setting '_remove' omap key 2026-02-26T03:06:28.115 INFO:tasks.workunit.client.0.trial123.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-26T03:06:28.115 INFO:tasks.workunit.client.0.trial123.stderr:Remove successful 2026-02-26T03:06:28.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:799: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.18886.out 2026-02-26T03:06:28.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:06:28.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:06:28.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-26T03:06:28.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:06:28.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-26T03:06:28.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.18886.out 2026-02-26T03:06:29.302 INFO:tasks.workunit.client.0.trial123.stdout:get_pg_num_history pg_num_history pg_num_history(e52 pg_nums {1={21=1,40=32}} deleted_pools ) 2026-02-26T03:06:29.303 INFO:tasks.workunit.client.0.trial123.stdout:Importing pgid 1.0 2026-02-26T03:06:29.303 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-26T03:06:29.303 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:06:29.303 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-26T03:06:29.303 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:06:29.304 INFO:tasks.workunit.client.0.trial123.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-26T03:06:29.304 INFO:tasks.workunit.client.0.trial123.stdout:snapset 0=[]:{} 2026-02-26T03:06:29.304 INFO:tasks.workunit.client.0.trial123.stdout:write_pg epoch 51 info 1.0( v 29'100 (29'90,29'100] local-lis/les=50/51 n=3 ec=21/21 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-26T03:06:29.304 INFO:tasks.workunit.client.0.trial123.stdout:Import successful 2026-02-26T03:06:29.838 INFO:tasks.workunit.client.0.trial123.stdout:reviving divergent 1 2026-02-26T03:06:29.838 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:801: TEST_divergent_3: echo 'reviving divergent 1' 2026-02-26T03:06:29.838 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:802: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:30.033 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:30.033 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953880+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660711+0000 0 0 2026-02-26T03:06:30.033 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953011+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660665+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953335+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660580+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952507+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660639+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952513+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660592+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660685+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952065+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660717+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952398+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660742+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952404+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660723+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952256+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660737+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952214+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660659+0000 0 0 2026-02-26T03:06:30.034 INFO:tasks.workunit.client.0.trial123.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951988+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660671+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952025+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660698+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952054+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660730+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951808+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660646+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953007+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660599+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953357+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660558+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952299+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660678+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-26T03:05:35.952352+0000 45'101 48:147 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660749+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952707+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660755+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952248+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660692+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951970+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660548+0000 0 0 2026-02-26T03:06:30.035 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953328+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660573+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952146+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660566+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953395+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660586+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953867+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660652+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953460+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660626+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953153+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660704+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952541+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660613+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.952273+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660620+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.951911+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660632+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-26T03:05:35.953081+0000 29'100 48:146 [0,2] 0 [0,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:06:25.660605+0000 0 0 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:30.036 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:30.037 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:803: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:06:30.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:06:30.046 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:06:30.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:06:30.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:06:30.048 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:06:30.048 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:06:30.049 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:06:30.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:06:30.050 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:06:30.050 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:06:30.050 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:06:30.050 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:06:30.072 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:30.064+0000 7f4b5a9ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:06:30.073 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:30.068+0000 7f4b5a9ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:06:30.073 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:30.068+0000 7f4b5a9ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:06:30.244 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:06:30.245 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:06:30.245 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:06:30.245 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:06:30.245 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:06:30.245 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:06:30.245 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:06:30.246 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:06:30.246 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:06:30.246 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:06:30.451 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/3367337205,v1:127.0.0.1:6819/3367337205] [v2:127.0.0.1:6820/3367337205,v1:127.0.0.1:6821/3367337205] exists,up 06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:06:30.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:06:30.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:06:30.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:06:30.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: wait_for_osd up 1 2026-02-26T03:06:30.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:06:30.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:06:30.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:06:30.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:06:30.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:06:30.453 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:06:30.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:06:30.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:06:30.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:06:30.651 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 31 last_clean_interval [10,29) [v2:127.0.0.1:6818/3367337205,v1:127.0.0.1:6819/3367337205] [v2:127.0.0.1:6820/3367337205,v1:127.0.0.1:6821/3367337205] exists,up 06feee01-2860-490b-b9ae-8a404b84c218 2026-02-26T03:06:30.652 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:06:30.652 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:06:30.652 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:06:30.652 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: sleep 20 2026-02-26T03:06:31.326 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:31.320+0000 7f4b5a9ab8c0 -1 Falling back to public interface 2026-02-26T03:06:34.325 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:06:34.320+0000 7f4b5a9ab8c0 -1 osd.1 52 log_to_monitors true 2026-02-26T03:06:50.654 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: get_asok_path osd.1 2026-02-26T03:06:50.654 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:06:50.654 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:06:50.655 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:06:50.655 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:06:50.655 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:06:50.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:06:50.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: CEPH_ARGS= 2026-02-26T03:06:50.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok dump_ops_in_flight 2026-02-26T03:06:50.716 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:06:50.717 INFO:tasks.workunit.client.0.trial123.stdout: "ops": [], 2026-02-26T03:06:50.717 INFO:tasks.workunit.client.0.trial123.stdout: "num_ops": 0 2026-02-26T03:06:50.717 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:06:50.722 INFO:tasks.workunit.client.0.trial123.stdout:allowing recovery 2026-02-26T03:06:50.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:809: TEST_divergent_3: echo 'allowing recovery' 2026-02-26T03:06:50.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:50.924 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:50.924 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.584911+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:40.628502+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.924 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.585213+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.924 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586160+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.924 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:46.327733+0000 29'100 64:178 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:46.327659+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T06:48:42.303357+0000 3 0 2026-02-26T03:06:50.924 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.584651+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:38.327516+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.9 5 0 0 0 0 3640 0 0 16 84 16 active+clean 2026-02-26T03:06:43.633124+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:43.633063+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T09:38:54.667742+0000 5 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:42.371617+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:42.371558+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T14:08:09.693989+0000 3 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.585179+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.6 5 0 0 0 0 3640 0 0 17 84 17 active+clean 2026-02-26T03:06:42.659677+0000 52'101 62:167 [0,1,2] 0 [0,1,2] 0 52'101 2026-02-26T03:06:42.659616+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T09:55:58.956728+0000 5 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586783+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.4 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:44.350628+0000 29'100 64:181 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:44.350569+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T12:27:01.193854+0000 2 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.3 5 0 0 0 0 3640 0 0 10 90 10 active+clean 2026-02-26T03:06:49.335120+0000 29'100 64:178 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:49.335038+0000 29'100 2026-02-26T03:06:49.335038+0000 0 0 periodic scrub scheduled @ 2026-02-27T14:43:17.973712+0000 5 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.2 1 0 0 0 0 728 0 0 16 84 16 active+clean 2026-02-26T03:06:43.640221+0000 29'100 62:172 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:43.640150+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T08:31:01.458578+0000 1 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 0 0 0 2184 0 0 14 86 14 active+clean 2026-02-26T03:06:40.359584+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:40.359506+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:30:51.763280+0000 3 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.1 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:44.683002+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:44.682939+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T07:14:40.246034+0000 2 0 2026-02-26T03:06:50.925 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:43.372594+0000 29'100 64:181 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:43.372494+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:46:10.290865+0000 2 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:37.348502+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:37.348431+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T14:25:45.033810+0000 3 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586776+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.e 4 0 0 0 0 3151 0 0 17 84 17 active+clean 2026-02-26T03:06:41.634762+0000 45'101 62:173 [0,1,2] 0 [0,1,2] 0 45'101 2026-02-26T03:06:41.634700+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T14:00:35.137001+0000 4 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.585188+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:41.349184+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:41.349109+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T13:56:42.632351+0000 3 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 0 0 0 2184 0 0 16 84 16 active+clean 2026-02-26T03:06:41.586308+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:41.586246+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T11:54:23.415712+0000 3 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.585356+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:35.372842+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:47.315001+0000 29'100 64:178 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:47.314892+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T08:54:17.169383+0000 3 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:45.326315+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:45.326255+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:23:38.702999+0000 3 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586312+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.585246+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.926 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.584694+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:36.380350+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:36.380266+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T08:27:27.515244+0000 3 0 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stdout:1.19 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:42.635401+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:42.635323+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T05:02:51.321325+0000 2 0 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 4 0 0 0 0 2912 0 0 16 84 16 active+clean 2026-02-26T03:06:44.624900+0000 29'100 62:172 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:44.624831+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T07:32:13.704239+0000 4 0 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 5 0 0 0 0 3640 0 0 10 90 10 active+clean 2026-02-26T03:06:48.356079+0000 29'100 64:180 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:48.355970+0000 29'100 2026-02-26T03:06:39.623755+0000 0 0 periodic scrub scheduled @ 2026-02-27T06:47:59.439717+0000 5 0 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:50.927 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:50.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-26T03:06:50.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-26T03:06:51.002 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:06:51.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-26T03:06:51.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-26T03:06:51.092 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:06:51.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-26T03:06:51.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-26T03:06:51.174 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:06:51.182 INFO:tasks.workunit.client.0.trial123.stdout:reading divergent objects 2026-02-26T03:06:51.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:817: TEST_divergent_3: echo 'reading divergent objects' 2026-02-26T03:06:51.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph pg dump pgs 2026-02-26T03:06:51.385 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:06:51.385 INFO:tasks.workunit.client.0.trial123.stdout:1.1f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.584911+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:40.628502+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.385 INFO:tasks.workunit.client.0.trial123.stdout:1.1e 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.585213+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.385 INFO:tasks.workunit.client.0.trial123.stdout:1.1d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586160+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.385 INFO:tasks.workunit.client.0.trial123.stdout:1.b 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:46.327733+0000 29'100 64:178 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:46.327659+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T06:48:42.303357+0000 3 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.a 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.584651+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:38.327516+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.9 5 0 0 0 0 3640 0 0 16 84 16 active+clean 2026-02-26T03:06:43.633124+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:43.633063+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T09:38:54.667742+0000 5 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.8 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:42.371617+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:42.371558+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T14:08:09.693989+0000 3 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.7 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.585179+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.6 5 0 0 0 0 3640 0 0 17 84 17 active+clean 2026-02-26T03:06:42.659677+0000 52'101 62:167 [0,1,2] 0 [0,1,2] 0 52'101 2026-02-26T03:06:42.659616+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T09:55:58.956728+0000 5 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.5 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586783+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.4 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:44.350628+0000 29'100 64:181 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:44.350569+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T12:27:01.193854+0000 2 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.3 5 0 0 0 0 3640 0 0 10 90 10 active+clean 2026-02-26T03:06:49.335120+0000 29'100 64:178 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:49.335038+0000 29'100 2026-02-26T03:06:49.335038+0000 0 0 periodic scrub scheduled @ 2026-02-27T14:43:17.973712+0000 5 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.2 1 0 0 0 0 728 0 0 16 84 16 active+clean 2026-02-26T03:06:43.640221+0000 29'100 62:172 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:43.640150+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T08:31:01.458578+0000 1 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.c 3 0 0 0 0 2184 0 0 14 86 14 active+clean 2026-02-26T03:06:40.359584+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:40.359506+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:30:51.763280+0000 3 0 2026-02-26T03:06:51.386 INFO:tasks.workunit.client.0.trial123.stdout:1.1 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:44.683002+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:44.682939+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T07:14:40.246034+0000 2 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.1c 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:43.372594+0000 29'100 64:181 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:43.372494+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T11:46:10.290865+0000 2 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.0 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:37.348502+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:37.348431+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T14:25:45.033810+0000 3 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586776+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.e 4 0 0 0 0 3151 0 0 17 84 17 active+clean 2026-02-26T03:06:41.634762+0000 45'101 62:173 [0,1,2] 0 [0,1,2] 0 45'101 2026-02-26T03:06:41.634700+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T14:00:35.137001+0000 4 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.585188+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.10 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:41.349184+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:41.349109+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T13:56:42.632351+0000 3 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.11 3 0 0 0 0 2184 0 0 16 84 16 active+clean 2026-02-26T03:06:41.586308+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:41.586246+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T11:54:23.415712+0000 3 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.12 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.585356+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:35.372842+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.13 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:47.315001+0000 29'100 64:178 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:47.314892+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T08:54:17.169383+0000 3 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.14 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:45.326315+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:45.326255+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T13:23:38.702999+0000 3 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.15 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:39.586312+0000 29'100 62:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.387 INFO:tasks.workunit.client.0.trial123.stdout:1.16 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:41.585246+0000 29'100 62:164 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout:1.17 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-26T03:06:43.584694+0000 29'100 64:168 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-26T03:04:44.308352+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 queued for scrub 0 0 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout:1.18 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-26T03:06:36.380350+0000 29'100 64:182 [1,0,2] 1 [1,0,2] 1 29'100 2026-02-26T03:06:36.380266+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T08:27:27.515244+0000 3 0 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout:1.19 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-26T03:06:42.635401+0000 29'100 62:151 [2,0,1] 2 [2,0,1] 2 29'100 2026-02-26T03:06:42.635323+0000 0'0 2026-02-26T03:04:44.308352+0000 0 0 periodic scrub scheduled @ 2026-02-27T05:02:51.321325+0000 2 0 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout:1.1a 4 0 0 0 0 2912 0 0 16 84 16 active+clean 2026-02-26T03:06:44.624900+0000 29'100 62:172 [0,1,2] 0 [0,1,2] 0 29'100 2026-02-26T03:06:44.624831+0000 0'0 2026-02-26T03:04:44.308352+0000 0 1 periodic scrub scheduled @ 2026-02-27T07:32:13.704239+0000 4 0 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout:1.1b 5 0 0 0 0 3640 0 0 10 90 10 active+clean 2026-02-26T03:06:48.356079+0000 29'100 64:180 [1,2,0] 1 [1,2,0] 1 29'100 2026-02-26T03:06:48.355970+0000 29'100 2026-02-26T03:06:39.623755+0000 0 0 periodic scrub scheduled @ 2026-02-27T06:47:59.439717+0000 5 0 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:06:51.388 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:06:51.395 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: expr 2 + 2 2026-02-26T03:06:51.396 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: seq 1 4 2026-02-26T03:06:51.397 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:06:51.397 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_1 td/divergent-priors/existing 2026-02-26T03:06:51.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:06:51.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_2 td/divergent-priors/existing 2026-02-26T03:06:51.447 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:06:51.447 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_3 td/divergent-priors/existing 2026-02-26T03:06:51.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:06:51.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_4 td/divergent-priors/existing 2026-02-26T03:06:51.503 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: seq 1 2 2026-02-26T03:06:51.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:06:51.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-26T03:06:51.524 INFO:tasks.workunit.client.0.trial123.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-26T03:06:51.526 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-26T03:06:51.526 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-26T03:06:51.552 INFO:tasks.workunit.client.0.trial123.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-26T03:06:51.554 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: rm -f td/divergent-priors/existing 2026-02-26T03:06:51.556 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-26T03:06:51.558 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:06:51.561 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 29'104 (29'1) modify 1:d858c035:::existing_1:head by client.4561.0:1 2026-02-26T03:04:57.180329+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:06:51.561 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 20 _merge_object_divergent_entries: keeping 29'104 (29'1) modify 1:d858c035:::existing_1:head by client.4561.0:1 2026-02-26T03:04:57.180329+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:06:51.561 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:06:51.561 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 29'1 first_divergent_update: 29'104 last_divergent_update: 29'104 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 29'106 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 29'104 (29'1) modify 1:d858c035:::existing_1:head by client.4561.0:1 2026-02-26T03:04:57.180329+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 29'1 <= info.log_tail 29'90 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 29'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4555.0:1 2026-02-26T03:04:57.177716+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 20 _merge_object_divergent_entries: keeping 29'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4555.0:1 2026-02-26T03:04:57.177716+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 29'102 last_divergent_update: 29'102 2026-02-26T03:06:51.562 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa045b640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 29'105 (29'3) delete 1:8881e6d8:::existing_3:head by client.4565.0:1 2026-02-26T03:04:57.183250+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 20 _merge_object_divergent_entries: keeping 29'105 (29'3) delete 1:8881e6d8:::existing_3:head by client.4565.0:1 2026-02-26T03:04:57.183250+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 29'3 first_divergent_update: 29'105 last_divergent_update: 29'105 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 29'106 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 29'105 (29'3) delete 1:8881e6d8:::existing_3:head by client.4565.0:1 2026-02-26T03:04:57.183250+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7eff9d455640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 29'3 <= info.log_tail 29'84 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 29'103 (29'2) modify 1:97db2a0f:::existing_2:head by client.4560.0:1 2026-02-26T03:04:57.180065+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:06:51.563 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 20 _merge_object_divergent_entries: keeping 29'103 (29'2) modify 1:97db2a0f:::existing_2:head by client.4560.0:1 2026-02-26T03:04:57.180065+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:06:51.564 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:06:51.564 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 29'2 first_divergent_update: 29'103 last_divergent_update: 29'103 2026-02-26T03:06:51.564 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-26T03:06:51.564 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:06:51.564 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 29'106 2026-02-26T03:06:51.564 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 29'103 (29'2) modify 1:97db2a0f:::existing_2:head by client.4560.0:1 2026-02-26T03:04:57.180065+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.184+0000 7effa145d640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 29'2 <= info.log_tail 29'89 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 29'106 (29'4) delete 1:70b3bf12:::existing_4:head by client.4567.0:1 2026-02-26T03:04:57.184935+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 20 _merge_object_divergent_entries: keeping 29'106 (29'4) delete 1:70b3bf12:::existing_4:head by client.4567.0:1 2026-02-26T03:04:57.184935+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 29'4 first_divergent_update: 29'106 last_divergent_update: 29'106 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: more recent entry found: 45'101 (29'4) modify 1:70b3bf12:::existing_4:head by client.4688.0:1 2026-02-26T03:05:49.314068+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false, already merged 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 29'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4554.0:1 2026-02-26T03:04:57.177253+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 20 _merge_object_divergent_entries: keeping 29'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4554.0:1 2026-02-26T03:04:57.177253+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:06:51.572 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:06:51.573 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 29'101 last_divergent_update: 29'101 2026-02-26T03:06:51.573 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-26T03:06:51.573 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:06:26.196+0000 7eff9ec58640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-26T03:06:51.573 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-26T03:06:51.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:06:51.578 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:837: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-26T03:06:51.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:837: TEST_divergent_3: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:06:51.582 INFO:tasks.workunit.client.0.trial123.stdout:success 2026-02-26T03:06:51.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:842: TEST_divergent_3: echo success 2026-02-26T03:06:51.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:844: TEST_divergent_3: rm td/divergent-priors/td/divergent-priors/exp.18886.out 2026-02-26T03:06:51.583 INFO:tasks.workunit.client.0.trial123.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.18886.out': No such file or directory 2026-02-26T03:06:51.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:846: TEST_divergent_3: delete_pool test 2026-02-26T03:06:51.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-26T03:06:51.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-26T03:06:51.827 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' does not exist 2026-02-26T03:06:51.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:847: TEST_divergent_3: kill_daemons td/divergent-priors 2026-02-26T03:06:51.836 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:06:51.836 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:06:51.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:06:51.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:06:51.838 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:07:00.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:07:00.155 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-26T03:07:00.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:07:00.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:07:00.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:07:00.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:07:00.156 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:07:00.157 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:07:00.157 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:07:00.157 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:07:00.163 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:07:00.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:07:00.164 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:07:00.164 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:07:00.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:07:00.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:07:00.166 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:07:00.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:07:00.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:07:00.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:07:00.168 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:07:00.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:07:00.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:07:00.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:07:00.199 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:07:00.200 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.200 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:07:00.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:07:00.202 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:07:00.202 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:07:00.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:07:00.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:07:00.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:07:00.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:07:00.205 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:07:00.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:07:00.206 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:07:00.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:07:00.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:07:00.207 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:07:00.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:07:00.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:07:00.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:07:00.209 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:07:00.210 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:07:00.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:07:00.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:07:00.212 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:07:00.212 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.212 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:07:00.214 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:07:00.214 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:07:00.214 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-26T03:07:00.215 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:07:00.215 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.215 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.215 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.18886 2026-02-26T03:07:00.217 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:07:00.218 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-26T03:07:00.218 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:07:00.218 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-26T03:07:00.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_ec td/divergent-priors 2026-02-26T03:07:00.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:228: TEST_divergent_ec: local dir=td/divergent-priors 2026-02-26T03:07:00.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: local dummyfile=/etc/fstab 2026-02-26T03:07:00.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:232: TEST_divergent_ec: local dummyfile2=/etc/resolv.conf 2026-02-26T03:07:00.219 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: local num_osds=3 2026-02-26T03:07:00.219 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: expr 3 - 1 2026-02-26T03:07:00.220 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: seq 0 2 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: local 'osds=0 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:236: TEST_divergent_ec: run_mon td/divergent-priors a 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:07:00.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:07:00.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:07:00.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-26T03:07:00.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-26T03:07:00.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:07:00.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:07:00.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:07:00.244 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:07:00.244 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.245 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.245 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:00.245 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:07:00.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:07:00.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:07:00.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:07:00.276 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:07:00.276 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:07:00.276 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:07:00.276 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:07:00.276 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:07:00.276 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:07:00.277 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:07:00.277 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.277 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.277 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T03:07:00.277 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:07:00.277 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get fsid 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:07:00.335 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:07:00.336 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:07:00.337 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.337 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.337 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-mon.a.asok 2026-02-26T03:07:00.337 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:07:00.337 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.18886/ceph-mon.a.asok config get mon_host 2026-02-26T03:07:00.391 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: run_mgr td/divergent-priors x 2026-02-26T03:07:00.391 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-26T03:07:00.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:07:00.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:07:00.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:07:00.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-26T03:07:00.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:07:00.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:07:00.503 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:07:00.503 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:07:00.503 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:07:00.503 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.503 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.504 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:00.504 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:07:00.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:07:00.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-26T03:07:00.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 0 2026-02-26T03:07:00.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:07:00.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:07:00.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:07:00.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:07:00.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:07:00.531 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:07:00.532 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:07:00.532 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:07:00.532 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1' 2026-02-26T03:07:00.532 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:07:00.533 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:07:00.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDUuJ9p8BNXIBAAEea9C22w0QR1QaitYWAOpA== 2026-02-26T03:07:00.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDUuJ9p8BNXIBAAEea9C22w0QR1QaitYWAOpA=="}' 2026-02-26T03:07:00.544 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 -i td/divergent-priors/0/new.json 2026-02-26T03:07:00.656 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:07:00.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-26T03:07:00.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUuJ9p8BNXIBAAEea9C22w0QR1QaitYWAOpA== --osd-uuid 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:07:00.684 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:00.676+0000 7f7e23b8f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:00.685 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:00.680+0000 7f7e23b8f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:00.686 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:00.680+0000 7f7e23b8f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:00.687 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:00.680+0000 7f7e23b8f8c0 -1 bdev(0x55e860783c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:07:00.687 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:00.680+0000 7f7e23b8f8c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-26T03:07:03.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-26T03:07:03.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:07:03.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:07:03.519 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:07:03.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:07:03.793 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:07:03.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:07:03.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:07:03.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:07:03.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:07:03.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:07:03.822 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:03.816+0000 7f537bfa98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:03.823 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:03.816+0000 7f537bfa98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:03.824 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:03.820+0000 7f537bfa98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:03.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:07:03.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:07:03.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:07:03.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:07:03.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:07:03.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:03.997 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:07:03.997 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:07:03.997 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:03.997 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:07:04.197 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:05.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:05.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:05.199 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:07:05.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:07:05.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:05.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:07:05.329 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:05.324+0000 7f537bfa98c0 -1 Falling back to public interface 2026-02-26T03:07:05.405 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:06.406 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:06.406 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:06.406 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:07:06.407 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:07:06.407 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:06.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:07:06.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:07.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:07.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:07.611 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:07:07.611 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:07:07.611 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:07.611 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:07:07.818 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:08.608 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:08.600+0000 7f537bfa98c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:07:08.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:08.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:08.819 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:07:08.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:07:08.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:08.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:07:09.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:10.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:10.031 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:10.031 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:07:10.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:07:10.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:10.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:07:10.234 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4073064297,v1:127.0.0.1:6803/4073064297] [v2:127.0.0.1:6804/4073064297,v1:127.0.0.1:6805/4073064297] exists,up 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:07:10.238 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:07:10.238 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:07:10.238 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:07:10.238 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-26T03:07:10.238 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 1 2026-02-26T03:07:10.238 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:07:10.239 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:10.240 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:07:10.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:07:10.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:07:10.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:07:10.242 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:07:10.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fa6691a7-52bf-4ea2-ac24-ed6cd404391d 2026-02-26T03:07:10.243 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 fa6691a7-52bf-4ea2-ac24-ed6cd404391d 2026-02-26T03:07:10.243 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 fa6691a7-52bf-4ea2-ac24-ed6cd404391d' 2026-02-26T03:07:10.243 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:07:10.253 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDeuJ9pwRz/DhAAiR+AFiiGrouf3+jidLgYeA== 2026-02-26T03:07:10.253 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDeuJ9pwRz/DhAAiR+AFiiGrouf3+jidLgYeA=="}' 2026-02-26T03:07:10.253 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fa6691a7-52bf-4ea2-ac24-ed6cd404391d -i td/divergent-priors/1/new.json 2026-02-26T03:07:10.454 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:07:10.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-26T03:07:10.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDeuJ9pwRz/DhAAiR+AFiiGrouf3+jidLgYeA== --osd-uuid fa6691a7-52bf-4ea2-ac24-ed6cd404391d 2026-02-26T03:07:10.492 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:10.484+0000 7f9a30eb98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:10.493 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:10.488+0000 7f9a30eb98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:10.494 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:10.488+0000 7f9a30eb98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:10.494 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:10.488+0000 7f9a30eb98c0 -1 bdev(0x563f5b91bc00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-26T03:07:10.495 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:10.488+0000 7f9a30eb98c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-26T03:07:12.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-26T03:07:12.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:07:12.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T03:07:12.766 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T03:07:12.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:07:13.043 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:07:13.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T03:07:13.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:07:13.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:07:13.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:07:13.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:07:13.072 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:13.068+0000 7f87592dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:13.074 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:13.068+0000 7f87592dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:13.075 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:13.068+0000 7f87592dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:13.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:07:13.249 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:07:13.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:13.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:07:13.448 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:14.450 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:14.450 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:14.451 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:07:14.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:07:14.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:14.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:07:14.573 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:14.568+0000 7f87592dd8c0 -1 Falling back to public interface 2026-02-26T03:07:14.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:15.652 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:15.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:15.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:07:15.653 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:07:15.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:15.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:07:15.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:16.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:16.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:16.860 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:07:16.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:07:16.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:16.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:07:17.061 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:17.628 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:17.624+0000 7f87592dd8c0 -1 osd.1 0 log_to_monitors true 2026-02-26T03:07:18.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:18.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:18.063 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:07:18.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:07:18.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:18.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:07:18.260 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3954289788,v1:127.0.0.1:6811/3954289788] [v2:127.0.0.1:6812/3954289788,v1:127.0.0.1:6813/3954289788] exists,up fa6691a7-52bf-4ea2-ac24-ed6cd404391d 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 2 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-26T03:07:18.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:07:18.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:07:18.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:07:18.267 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:07:18.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:07:18.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 76430717-38c9-43dd-9de4-8eee86d1bcbb' 2026-02-26T03:07:18.268 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:07:18.269 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:07:18.279 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDmuJ9pGouREBAAemIZeK3pydcPGpQb/Cc/tw== 2026-02-26T03:07:18.279 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDmuJ9pGouREBAAemIZeK3pydcPGpQb/Cc/tw=="}' 2026-02-26T03:07:18.279 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 76430717-38c9-43dd-9de4-8eee86d1bcbb -i td/divergent-priors/2/new.json 2026-02-26T03:07:18.498 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:07:18.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-26T03:07:18.508 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDmuJ9pGouREBAAemIZeK3pydcPGpQb/Cc/tw== --osd-uuid 76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:07:18.528 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:18.520+0000 7fb8b75438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:18.529 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:18.524+0000 7fb8b75438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:18.530 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:18.524+0000 7fb8b75438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:18.531 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:18.524+0000 7fb8b75438c0 -1 bdev(0x55d81e9efc00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-26T03:07:18.531 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:18.524+0000 7fb8b75438c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-26T03:07:20.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-26T03:07:20.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:07:20.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T03:07:20.830 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T03:07:20.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:07:21.119 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:07:21.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T03:07:21.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:07:21.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:07:21.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:07:21.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:07:21.148 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:21.140+0000 7ff17d4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:21.149 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:21.144+0000 7ff17d4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:21.150 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:21.144+0000 7ff17d4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:07:21.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T03:07:21.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:07:21.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:07:21.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:07:21.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:07:21.316 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:07:21.317 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:21.317 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:07:21.317 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:21.317 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:07:21.521 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:22.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:22.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:22.523 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:07:22.524 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:07:22.524 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:22.524 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:07:22.729 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:22.874 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:22.867+0000 7ff17d4538c0 -1 Falling back to public interface 2026-02-26T03:07:23.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:23.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:23.731 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:07:23.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:07:23.732 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:23.732 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:07:23.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:24.934 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:24.934 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:24.935 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:07:24.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:07:24.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:24.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:07:25.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:25.872 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:07:25.871+0000 7ff17d4538c0 -1 osd.2 0 log_to_monitors true 2026-02-26T03:07:26.140 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:26.140 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:26.141 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:07:26.141 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:07:26.141 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:26.141 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:07:26.341 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:07:27.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:07:27.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:07:27.344 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:07:27.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:07:27.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:07:27.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:07:27.544 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/309982560,v1:127.0.0.1:6819/309982560] [v2:127.0.0.1:6820/309982560,v1:127.0.0.1:6821/309982560] exists,up 76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:07:27.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:07:27.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:07:27.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:07:27.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:243: TEST_divergent_ec: ceph osd set noout 2026-02-26T03:07:27.853 INFO:tasks.workunit.client.0.trial123.stderr:noout is set 2026-02-26T03:07:27.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:244: TEST_divergent_ec: ceph osd set noin 2026-02-26T03:07:28.158 INFO:tasks.workunit.client.0.trial123.stderr:noin is set 2026-02-26T03:07:28.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:245: TEST_divergent_ec: ceph osd set nodown 2026-02-26T03:07:28.465 INFO:tasks.workunit.client.0.trial123.stderr:nodown is set 2026-02-26T03:07:28.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:246: TEST_divergent_ec: create_ec_pool test true k=2 m=1 2026-02-26T03:07:28.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=test 2026-02-26T03:07:28.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-26T03:07:28.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-26T03:07:28.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-26T03:07:28.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-26T03:07:28.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool test 1 1 erasure myprofile 2026-02-26T03:07:28.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 erasure myprofile 2026-02-26T03:07:29.117 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' already exists 2026-02-26T03:07:29.127 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T03:07:30.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-26T03:07:30.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set test allow_ec_overwrites true 2026-02-26T03:07:30.434 INFO:tasks.workunit.client.0.trial123.stderr:set pool 1 allow_ec_overwrites to true 2026-02-26T03:07:30.447 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-26T03:07:30.448 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:07:30.448 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:07:30.448 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:07:30.448 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:07:30.449 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:07:30.449 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:07:30.449 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:07:30.449 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:07:30.449 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:07:30.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:07:30.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:07:30.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:07:30.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:07:30.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:07:30.566 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:07:30.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:07:30.768 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:07:30.768 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:07:30.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:07:30.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:30.769 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:07:30.846 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-26T03:07:30.846 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-26T03:07:30.846 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-26T03:07:30.846 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:30.846 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:07:30.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-26T03:07:30.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-26T03:07:30.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-26T03:07:30.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:30.926 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:07:31.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-26T03:07:31.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-26T03:07:31.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-60129542147' 2026-02-26T03:07:31.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:31.010 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-26T03:07:31.010 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:31.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:07:31.012 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-26T03:07:31.012 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:31.013 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-26T03:07:31.013 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-26T03:07:31.013 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836487 2026-02-26T03:07:31.014 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:31.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-26T03:07:31.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:32.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:07:32.223 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:32.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-26T03:07:32.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:33.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:07:33.424 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:33.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-26T03:07:33.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:33.629 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-26T03:07:33.629 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:33.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:07:33.631 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-26T03:07:33.631 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:33.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-26T03:07:33.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-26T03:07:33.632 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672965 2026-02-26T03:07:33.633 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:07:33.835 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672965 -lt 42949672965 2026-02-26T03:07:33.835 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:33.836 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-26T03:07:33.836 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:33.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:07:33.838 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-26T03:07:33.838 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:33.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-26T03:07:33.840 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-26T03:07:33.840 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542147 2026-02-26T03:07:33.840 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:07:34.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542147 2026-02-26T03:07:34.054 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:07:34.054 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:07:34.054 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:07:34.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:07:34.326 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:07:34.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:07:34.518 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:07:34.518 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:07:34.518 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:248: TEST_divergent_ec: flush_pg_stats 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:07:34.773 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:07:34.981 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:07:34.981 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:07:34.981 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:07:34.981 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:07:34.981 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:34.981 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:07:35.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836490 2026-02-26T03:07:35.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836490 2026-02-26T03:07:35.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836490' 2026-02-26T03:07:35.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:35.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:07:35.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-26T03:07:35.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-26T03:07:35.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836490 1-42949672968' 2026-02-26T03:07:35.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:35.143 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:07:35.223 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-26T03:07:35.223 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-26T03:07:35.223 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836490 1-42949672968 2-60129542150' 2026-02-26T03:07:35.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:35.224 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836490 2026-02-26T03:07:35.224 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:35.225 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:07:35.225 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836490 2026-02-26T03:07:35.226 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:35.227 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836490 2026-02-26T03:07:35.227 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-26T03:07:35.227 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836490 2026-02-26T03:07:35.228 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:35.433 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-26T03:07:35.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:36.435 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:07:36.435 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:36.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-26T03:07:36.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:37.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:07:37.641 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:37.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-26T03:07:37.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:37.843 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-26T03:07:37.844 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:37.846 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:07:37.847 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-26T03:07:37.847 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:37.848 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-26T03:07:37.848 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-26T03:07:37.848 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672968 2026-02-26T03:07:37.849 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:07:38.049 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-26T03:07:38.049 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:38.050 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-26T03:07:38.050 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:38.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:07:38.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-26T03:07:38.052 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:38.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-26T03:07:38.053 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542150 2026-02-26T03:07:38.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-26T03:07:38.054 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:07:38.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-26T03:07:38.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:249: TEST_divergent_ec: wait_for_clean 2026-02-26T03:07:38.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:07:38.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:07:38.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:07:38.269 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:07:38.270 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:07:38.270 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:07:38.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:07:38.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:07:38.270 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:07:38.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:07:38.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:07:38.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:07:38.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:07:38.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:07:38.382 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:07:38.578 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:07:38.578 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:07:38.578 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:07:38.578 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:07:38.578 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:38.578 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:07:38.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-26T03:07:38.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-26T03:07:38.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-26T03:07:38.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:38.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:07:38.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672970 2026-02-26T03:07:38.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672970 2026-02-26T03:07:38.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672970' 2026-02-26T03:07:38.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:38.738 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:07:38.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-26T03:07:38.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-26T03:07:38.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672970 2-60129542153' 2026-02-26T03:07:38.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:38.820 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-26T03:07:38.820 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:38.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:07:38.821 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-26T03:07:38.821 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:38.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-26T03:07:38.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-26T03:07:38.823 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836492 2026-02-26T03:07:38.824 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:39.033 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-26T03:07:39.033 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:39.034 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672970 2026-02-26T03:07:39.034 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:39.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:07:39.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672970 2026-02-26T03:07:39.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:39.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672970 2026-02-26T03:07:39.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-26T03:07:39.038 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672970 2026-02-26T03:07:39.038 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:07:39.244 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672970 2026-02-26T03:07:39.244 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:39.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-26T03:07:39.244 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:39.246 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:07:39.246 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-26T03:07:39.246 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:39.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-26T03:07:39.248 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542153 2026-02-26T03:07:39.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-26T03:07:39.248 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:07:39.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542151 -lt 60129542153 2026-02-26T03:07:39.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:40.447 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:07:40.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:07:40.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542151 -lt 60129542153 2026-02-26T03:07:40.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:41.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:07:41.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:07:41.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-26T03:07:41.845 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:07:41.845 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:07:41.846 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:07:42.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:07:42.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:07:42.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:07:42.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:07:42.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:07:42.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:07:42.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:07:42.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:07:42.297 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:07:42.297 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:07:42.298 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:07:42.298 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:07:42.555 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:07:42.555 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:07:42.555 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:07:42.555 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: ceph pg dump pgs --format=json 2026-02-26T03:07:42.556 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: jq '.pg_stats[0].up_primary' 2026-02-26T03:07:42.744 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:07:42.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: local divergent=1 2026-02-26T03:07:42.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:253: TEST_divergent_ec: echo 'primary and soon to be divergent is 1' 2026-02-26T03:07:42.756 INFO:tasks.workunit.client.0.trial123.stdout:primary and soon to be divergent is 1 2026-02-26T03:07:42.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:254: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:07:42.942 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:07:42.942 INFO:tasks.workunit.client.0.trial123.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-26T03:07:29.668174+0000 0'0 27:19 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:39:34.168530+0000 0 0 2026-02-26T03:07:42.942 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:07:42.942 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:07:42.943 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: local non_divergent= 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 0 = 1 ']' 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0' 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 1 = 1 ']' 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:259: TEST_divergent_ec: continue 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-26T03:07:42.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 2 = 1 ']' 2026-02-26T03:07:42.953 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0 2' 2026-02-26T03:07:42.953 INFO:tasks.workunit.client.0.trial123.stdout:writing initial objects 2026-02-26T03:07:42.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:264: TEST_divergent_ec: echo 'writing initial objects' 2026-02-26T03:07:42.954 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: seq 1 100 2026-02-26T03:07:42.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:42.954 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_1 /etc/fstab 2026-02-26T03:07:42.976 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:42.976 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_2 /etc/fstab 2026-02-26T03:07:43.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_3 /etc/fstab 2026-02-26T03:07:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_4 /etc/fstab 2026-02-26T03:07:43.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_5 /etc/fstab 2026-02-26T03:07:43.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_6 /etc/fstab 2026-02-26T03:07:43.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_7 /etc/fstab 2026-02-26T03:07:43.141 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.142 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_8 /etc/fstab 2026-02-26T03:07:43.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_9 /etc/fstab 2026-02-26T03:07:43.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_10 /etc/fstab 2026-02-26T03:07:43.225 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_11 /etc/fstab 2026-02-26T03:07:43.253 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.253 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_12 /etc/fstab 2026-02-26T03:07:43.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_13 /etc/fstab 2026-02-26T03:07:43.309 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_14 /etc/fstab 2026-02-26T03:07:43.337 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.337 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_15 /etc/fstab 2026-02-26T03:07:43.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_16 /etc/fstab 2026-02-26T03:07:43.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_17 /etc/fstab 2026-02-26T03:07:43.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_18 /etc/fstab 2026-02-26T03:07:43.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_19 /etc/fstab 2026-02-26T03:07:43.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_20 /etc/fstab 2026-02-26T03:07:43.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_21 /etc/fstab 2026-02-26T03:07:43.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_22 /etc/fstab 2026-02-26T03:07:43.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_23 /etc/fstab 2026-02-26T03:07:43.586 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.586 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_24 /etc/fstab 2026-02-26T03:07:43.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_25 /etc/fstab 2026-02-26T03:07:43.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_26 /etc/fstab 2026-02-26T03:07:43.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.665 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_27 /etc/fstab 2026-02-26T03:07:43.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.690 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_28 /etc/fstab 2026-02-26T03:07:43.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_29 /etc/fstab 2026-02-26T03:07:43.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_30 /etc/fstab 2026-02-26T03:07:43.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_31 /etc/fstab 2026-02-26T03:07:43.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_32 /etc/fstab 2026-02-26T03:07:43.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_33 /etc/fstab 2026-02-26T03:07:43.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_34 /etc/fstab 2026-02-26T03:07:43.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_35 /etc/fstab 2026-02-26T03:07:43.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_36 /etc/fstab 2026-02-26T03:07:43.941 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.942 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_37 /etc/fstab 2026-02-26T03:07:43.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_38 /etc/fstab 2026-02-26T03:07:43.998 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:43.998 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_39 /etc/fstab 2026-02-26T03:07:44.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.026 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_40 /etc/fstab 2026-02-26T03:07:44.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_41 /etc/fstab 2026-02-26T03:07:44.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_42 /etc/fstab 2026-02-26T03:07:44.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_43 /etc/fstab 2026-02-26T03:07:44.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_44 /etc/fstab 2026-02-26T03:07:44.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.165 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_45 /etc/fstab 2026-02-26T03:07:44.193 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.193 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_46 /etc/fstab 2026-02-26T03:07:44.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_47 /etc/fstab 2026-02-26T03:07:44.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_48 /etc/fstab 2026-02-26T03:07:44.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_49 /etc/fstab 2026-02-26T03:07:44.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_50 /etc/fstab 2026-02-26T03:07:44.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_51 /etc/fstab 2026-02-26T03:07:44.361 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.361 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_52 /etc/fstab 2026-02-26T03:07:44.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_53 /etc/fstab 2026-02-26T03:07:44.417 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.417 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_54 /etc/fstab 2026-02-26T03:07:44.445 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.445 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_55 /etc/fstab 2026-02-26T03:07:44.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_56 /etc/fstab 2026-02-26T03:07:44.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_57 /etc/fstab 2026-02-26T03:07:44.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_58 /etc/fstab 2026-02-26T03:07:44.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_59 /etc/fstab 2026-02-26T03:07:44.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_60 /etc/fstab 2026-02-26T03:07:44.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.614 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_61 /etc/fstab 2026-02-26T03:07:44.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_62 /etc/fstab 2026-02-26T03:07:44.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_63 /etc/fstab 2026-02-26T03:07:44.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_64 /etc/fstab 2026-02-26T03:07:44.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_65 /etc/fstab 2026-02-26T03:07:44.753 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.753 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_66 /etc/fstab 2026-02-26T03:07:44.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.781 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_67 /etc/fstab 2026-02-26T03:07:44.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_68 /etc/fstab 2026-02-26T03:07:44.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.837 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_69 /etc/fstab 2026-02-26T03:07:44.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.865 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_70 /etc/fstab 2026-02-26T03:07:44.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_71 /etc/fstab 2026-02-26T03:07:44.921 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.921 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_72 /etc/fstab 2026-02-26T03:07:44.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_73 /etc/fstab 2026-02-26T03:07:44.977 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:44.977 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_74 /etc/fstab 2026-02-26T03:07:45.005 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_75 /etc/fstab 2026-02-26T03:07:45.033 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.034 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_76 /etc/fstab 2026-02-26T03:07:45.062 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.062 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_77 /etc/fstab 2026-02-26T03:07:45.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_78 /etc/fstab 2026-02-26T03:07:45.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_79 /etc/fstab 2026-02-26T03:07:45.139 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.139 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_80 /etc/fstab 2026-02-26T03:07:45.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_81 /etc/fstab 2026-02-26T03:07:45.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_82 /etc/fstab 2026-02-26T03:07:45.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_83 /etc/fstab 2026-02-26T03:07:45.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_84 /etc/fstab 2026-02-26T03:07:45.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_85 /etc/fstab 2026-02-26T03:07:45.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_86 /etc/fstab 2026-02-26T03:07:45.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_87 /etc/fstab 2026-02-26T03:07:45.362 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.362 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_88 /etc/fstab 2026-02-26T03:07:45.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.390 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_89 /etc/fstab 2026-02-26T03:07:45.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.410 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_90 /etc/fstab 2026-02-26T03:07:45.438 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.438 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_91 /etc/fstab 2026-02-26T03:07:45.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_92 /etc/fstab 2026-02-26T03:07:45.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_93 /etc/fstab 2026-02-26T03:07:45.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_94 /etc/fstab 2026-02-26T03:07:45.540 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.541 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_95 /etc/fstab 2026-02-26T03:07:45.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_96 /etc/fstab 2026-02-26T03:07:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.597 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_97 /etc/fstab 2026-02-26T03:07:45.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.618 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_98 /etc/fstab 2026-02-26T03:07:45.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2026-02-26T03:07:45.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-26T03:07:45.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2026-02-26T03:07:45.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:07:45.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: wait_for_clean 2026-02-26T03:07:45.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:07:45.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:07:45.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:07:45.695 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:07:45.696 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:07:45.696 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:07:45.696 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:07:45.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:07:45.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:07:45.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:07:45.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:07:45.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:07:45.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:07:45.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:07:45.770 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:07:45.964 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:07:45.965 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:07:45.965 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:07:45.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:07:45.965 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:45.965 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:07:46.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836496 2026-02-26T03:07:46.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836496 2026-02-26T03:07:46.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496' 2026-02-26T03:07:46.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:46.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:07:46.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672974 2026-02-26T03:07:46.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672974 2026-02-26T03:07:46.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-42949672974' 2026-02-26T03:07:46.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:07:46.126 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:07:46.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542156 2026-02-26T03:07:46.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542156 2026-02-26T03:07:46.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-42949672974 2-60129542156' 2026-02-26T03:07:46.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:46.207 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836496 2026-02-26T03:07:46.207 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:46.208 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:07:46.209 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836496 2026-02-26T03:07:46.209 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:46.210 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836496 2026-02-26T03:07:46.211 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 21474836496 2026-02-26T03:07:46.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-26T03:07:46.211 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:46.415 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836496 2026-02-26T03:07:46.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:07:47.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:07:47.417 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:07:47.624 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-26T03:07:47.624 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:47.624 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672974 2026-02-26T03:07:47.625 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:47.626 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:07:47.627 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672974 2026-02-26T03:07:47.628 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:47.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672974 2026-02-26T03:07:47.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672974' 2026-02-26T03:07:47.629 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 42949672974 2026-02-26T03:07:47.630 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:07:47.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672974 -lt 42949672974 2026-02-26T03:07:47.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:07:47.827 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542156 2026-02-26T03:07:47.827 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:07:47.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:07:47.829 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542156 2026-02-26T03:07:47.829 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:07:47.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542156 2026-02-26T03:07:47.831 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 60129542156 2026-02-26T03:07:47.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-26T03:07:47.831 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:07:48.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-26T03:07:48.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:07:48.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:07:48.040 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:07:48.301 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:07:48.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:07:48.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:07:48.504 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:07:48.505 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:07:48.505 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:07:48.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:07:48.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:07:48.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:07:48.773 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: get_pg test existing_1 2026-02-26T03:07:48.774 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-26T03:07:48.774 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-26T03:07:48.774 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-26T03:07:48.774 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-26T03:07:48.981 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: local pgid=1.0 2026-02-26T03:07:48.982 INFO:tasks.workunit.client.0.trial123.stdout:blackholing osds 0 2 2026-02-26T03:07:48.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:276: TEST_divergent_ec: echo 'blackholing osds 0 2' 2026-02-26T03:07:48.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:07:49.171 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:07:49.171 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-26T03:07:29.668174+0000 27'100 27:119 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T03:39:34.168530+0000 0 0 2026-02-26T03:07:49.172 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:07:49.172 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:07:49.172 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:07:49.180 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.0 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:49.181 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:07:49.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2026-02-26T03:07:49.182 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-26T03:07:49.242 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:07:49.242 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:07:49.242 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:07:49.247 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.2 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:07:49.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:07:49.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2026-02-26T03:07:49.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-26T03:07:49.304 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:07:49.304 INFO:tasks.workunit.client.0.trial123.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-26T03:07:49.305 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:07:49.310 INFO:tasks.workunit.client.0.trial123.stdout:writing divergent object 2026-02-26T03:07:49.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:284: TEST_divergent_ec: echo 'writing divergent object' 2026-02-26T03:07:49.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:286: TEST_divergent_ec: sleep 1 2026-02-26T03:07:49.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:285: TEST_divergent_ec: rados -p test put existing_100 /etc/resolv.conf 2026-02-26T03:07:50.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: rados -p test mksnap snap1 2026-02-26T03:07:50.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:287: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2026-02-26T03:07:50.384 INFO:tasks.workunit.client.0.trial123.stdout:created pool test snap snap1 2026-02-26T03:07:50.387 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:290: TEST_divergent_ec: sleep 10 2026-02-26T03:07:50.388 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: expr 100 - 1 2026-02-26T03:07:50.389 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2026-02-26T03:08:00.388 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:291: TEST_divergent_ec: killall -9 rados 2026-02-26T03:08:00.433 INFO:tasks.workunit.client.0.trial123.stderr:rados: no process found 2026-02-26T03:08:00.434 INFO:tasks.workunit.client.0.trial123.stdout:killing all the osds 2026-02-26T03:08:00.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:294: TEST_divergent_ec: echo 'killing all the osds' 2026-02-26T03:08:00.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:295: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:08:00.638 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:08:00.638 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 0 0 0 73528 0 0 14 90 14 active+clean 2026-02-26T03:07:29.668174+0000 28'104 28:125 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T09:29:52.572177+0000 0 0 2026-02-26T03:08:00.638 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:08:00.638 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:08:00.638 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:08:00.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd 2026-02-26T03:08:00.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:08:00.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:08:00.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:08:00.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:08:00.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:08:00.757 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:08:00.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-26T03:08:00.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.0 2026-02-26T03:08:00.993 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:08:01.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-26T03:08:01.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.1 2026-02-26T03:08:01.263 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:08:01.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-26T03:08:01.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.2 2026-02-26T03:08:01.531 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:08:01.541 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:01.541 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.0 2026-02-26T03:08:01.798 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already out. 2026-02-26T03:08:01.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:01.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.2 2026-02-26T03:08:02.059 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already out. 2026-02-26T03:08:02.069 INFO:tasks.workunit.client.0.trial123.stdout:bringing up non_divergent 0 2 2026-02-26T03:08:02.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:307: TEST_divergent_ec: echo 'bringing up non_divergent 0 2' 2026-02-26T03:08:02.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:08:02.265 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:08:02.265 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 0 0 0 73528 0 0 14 90 14 stale+active+clean 2026-02-26T03:07:29.668174+0000 28'104 28:125 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T09:29:52.572177+0000 0 0 2026-02-26T03:08:02.265 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:08:02.265 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:08:02.265 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:08:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:08:02.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:08:02.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:08:02.277 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:08:02.277 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:08:02.278 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:08:02.278 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-26T03:08:02.279 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:08:02.279 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:08:02.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:08:02.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:08:02.304 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:02.303+0000 7f33d57a38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:02.305 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:02.303+0000 7f33d57a38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:02.306 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:02.303+0000 7f33d57a38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:02.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:08:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:08:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:08:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:08:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:08:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:02.483 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:08:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:08:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:08:02.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:03.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:03.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:03.684 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:08:03.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:08:03.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:03.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:08:03.825 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:03.823+0000 7f33d57a38c0 -1 Falling back to public interface 2026-02-26T03:08:03.883 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:04.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:04.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:04.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:08:04.886 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:08:04.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:04.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:08:05.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:06.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:06.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:06.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:08:06.091 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:08:06.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:06.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:08:06.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:06.929 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:06.927+0000 7f33d57a38c0 -1 osd.0 27 log_to_monitors true 2026-02-26T03:08:07.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:07.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:07.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:08:07.292 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:08:07.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:07.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:08:07.493 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6802/4004398083,v1:127.0.0.1:6803/4004398083] [v2:127.0.0.1:6804/4004398083,v1:127.0.0.1:6805/4004398083] exists,up 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:08:07.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:08:07.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:08:07.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:08:07.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:07.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-26T03:08:07.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:08:07.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:08:07.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:08:07.497 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:08:07.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:08:07.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:08:07.497 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-26T03:08:07.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:08:07.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:08:07.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:08:07.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:08:07.523 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:07.519+0000 7fb8004108c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:07.525 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:07.523+0000 7fb8004108c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:07.526 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:07.523+0000 7fb8004108c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:07.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:08:07.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:08:07.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:08:07.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:08:07.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:08:07.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:07.699 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:08:07.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:08:07.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:07.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:08:07.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:08.749 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:08.747+0000 7fb8004108c0 -1 Falling back to public interface 2026-02-26T03:08:08.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:08.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:08.897 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:08:08.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:08:08.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:08.898 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:08:09.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:10.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:10.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:10.096 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:08:10.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:08:10.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:10.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:08:10.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:11.294 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:11.294 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:11.295 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:08:11.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:08:11.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:11.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:08:11.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:08:11.855 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:11.851+0000 7fb8004108c0 -1 osd.2 27 log_to_monitors true 2026-02-26T03:08:12.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:08:12.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:12.495 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:08:12.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:08:12.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:12.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:08:12.695 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up out weight 0 up_from 37 up_thru 0 down_at 31 last_clean_interval [14,27) [v2:127.0.0.1:6810/4113961987,v1:127.0.0.1:6811/4113961987] [v2:127.0.0.1:6812/4113961987,v1:127.0.0.1:6813/4113961987] exists,up 76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:08:12.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:08:12.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:08:12.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:08:12.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:12.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.0 2026-02-26T03:08:12.943 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already in. 2026-02-26T03:08:12.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:12.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.2 2026-02-26T03:08:13.249 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already in. 2026-02-26T03:08:13.259 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:318: TEST_divergent_ec: sleep 5 2026-02-26T03:08:18.271 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: expr 2 + 2 2026-02-26T03:08:18.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: objname=existing_4 2026-02-26T03:08:18.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:323: TEST_divergent_ec: echo 'writing non-divergent object existing_4' 2026-02-26T03:08:18.271 INFO:tasks.workunit.client.0.trial123.stdout:writing non-divergent object existing_4 2026-02-26T03:08:18.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:324: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:08:18.455 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:08:18.455 INFO:tasks.workunit.client.0.trial123.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 unknown 2026-02-26T03:08:12.947625+0000 27'100 38:121 [NONE,0,NONE] 0 [NONE,0,NONE] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 no scrub is scheduled 0 0 2026-02-26T03:08:18.455 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:08:18.455 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:08:18.456 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:08:18.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:325: TEST_divergent_ec: rados -p test put existing_4 /etc/resolv.conf 2026-02-26T03:08:29.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:08:29.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: wait_for_clean 2026-02-26T03:08:29.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:08:29.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:08:29.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:08:29.106 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:08:29.107 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:08:29.107 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:08:29.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:08:29.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:08:29.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:08:29.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:08:29.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:08:29.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:08:29.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:08:29.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:08:29.169 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:08:29.364 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:08:29.364 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:08:29.365 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:08:29.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:08:29.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:08:29.365 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:08:29.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855367 2026-02-26T03:08:29.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855367 2026-02-26T03:08:29.446 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855367' 2026-02-26T03:08:29.447 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:08:29.447 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:08:29.506 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:08:29.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:08:29.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:08:29.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:08:29.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:08:29.509 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:08:29.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789958 2026-02-26T03:08:29.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789958 2026-02-26T03:08:29.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855367 2-158913789958' 2026-02-26T03:08:29.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:08:29.586 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855367 2026-02-26T03:08:29.586 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:08:29.587 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:08:29.588 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855367 2026-02-26T03:08:29.588 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:08:29.589 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855367 2026-02-26T03:08:29.589 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855367' 2026-02-26T03:08:29.590 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 150323855367 2026-02-26T03:08:29.590 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:08:29.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855365 -lt 150323855367 2026-02-26T03:08:29.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:08:30.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:08:30.806 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:08:31.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855367 -lt 150323855367 2026-02-26T03:08:31.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:08:31.007 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-158913789958 2026-02-26T03:08:31.007 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:08:31.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:08:31.009 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-158913789958 2026-02-26T03:08:31.009 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:08:31.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789958 2026-02-26T03:08:31.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 158913789958' 2026-02-26T03:08:31.011 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 158913789958 2026-02-26T03:08:31.011 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:08:31.226 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789958 -lt 158913789958 2026-02-26T03:08:31.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:08:31.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:31.227 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:31.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:08:31.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:31.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:31.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:31.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:31.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:31.496 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:31.496 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:31.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:31.696 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:31.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:31.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:31.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:31.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:08:31.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:08:31.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:08:31.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:31.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:08:32.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:32.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:32.063 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:32.063 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:32.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:32.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:32.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:32.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:32.267 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:32.268 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:32.268 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:32.268 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:32.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:32.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:32.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:32.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:32.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:32.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:32.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:32.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:32.530 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:32.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:32.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:32.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:08:32.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:32.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:08:33.000 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:33.000 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:33.000 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:33.000 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:33.001 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:33.001 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:33.001 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:33.001 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:33.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:33.199 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:33.200 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:33.200 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:33.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:33.458 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:33.458 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:33.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:33.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:33.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:08:33.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:33.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:08:34.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:34.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:34.117 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:34.117 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:34.117 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:34.118 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:34.118 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:34.118 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:34.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:34.314 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:34.314 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:34.314 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:34.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:34.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:34.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:34.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:34.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:34.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:34.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:34.573 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:34.573 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:34.833 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:34.833 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:34.833 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:08:34.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:34.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:08:35.634 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:35.634 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:35.635 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:35.635 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:35.635 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:35.635 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:35.636 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:35.636 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:35.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:35.829 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:35.829 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:35.829 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:36.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:36.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:36.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:36.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:36.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:36.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:36.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:36.088 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:36.088 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:36.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:36.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:36.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:08:36.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:36.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:08:37.944 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:37.944 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:37.945 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:37.946 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:37.946 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:37.946 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:37.946 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:37.946 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:38.140 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:38.141 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:38.141 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:38.141 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:38.407 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:38.408 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:38.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:38.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:38.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:08:38.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:38.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:08:41.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:41.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:41.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:41.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:41.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:41.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:41.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:41.877 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:42.070 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:42.071 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:42.071 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:42.071 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:42.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:42.330 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:42.330 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:42.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:42.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:42.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:08:42.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:42.584 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:08:48.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:48.985 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:48.986 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:48.986 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:48.986 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:48.987 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:48.987 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:48.987 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:49.183 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:49.184 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:49.184 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:49.185 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:49.453 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:49.454 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:49.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:49.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:49.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:08:49.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:08:49.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:08:57.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:08:57.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:08:57.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:08:57.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:08:57.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:08:57.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:08:57.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:08:57.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:08:57.223 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:08:57.224 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:08:57.224 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:08:57.224 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:08:57.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:08:57.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:08:57.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:08:57.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:08:57.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:08:57.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:08:57.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:08:57.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:08:57.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:08:57.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:08:57.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:08:57.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:08:57.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:08:58.009 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:08:58.009 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "1031cb3a-2959-4a4f-83c3-ab3ff247c705", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:08:58.008985+0000", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:08:58.010 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:08:58.011 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:08:58.012 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:08:58.013 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.014 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:08:58.015 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:07:00.236386Z", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:07:00.236386Z", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:08:58.016 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:08:58.017 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.018 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:08:58.019 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 42, 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:07:00.317191+0000", 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:08:13.301335+0000", 2026-02-26T03:08:58.020 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:08:12.011139+0000", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:08:13.198836+0000", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:08:58.021 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 10, 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:08:58.022 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T03:07:28.977995+0000", 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 16389, 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "type": 3, 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:08:58.023 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 1, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "off", 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 1, 2026-02-26T03:08:58.024 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 1, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 1, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 1, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "28", 2026-02-26T03:08:58.025 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "0", 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "pool", 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 1, 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 28, 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [ 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "snapid": 1, 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "stamp": "2026-02-26T03:07:50.332149+0000", 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: "name": "snap1" 2026-02-26T03:08:58.026 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:08:58.027 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "myprofile", 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:08:58.028 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 8192, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:08:58.029 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {} 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1", 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:08:58.030 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 5, 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 27, 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 35, 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 41, 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 29, 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:08:58.031 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.032 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.033 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:08:58.034 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.035 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4004398083 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/4004398083", 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/4004398083", 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/4004398083", 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/4004398083", 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:08:58.036 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "fa6691a7-52bf-4ea2-ac24-ed6cd404391d", 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:08:58.037 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 10, 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 23, 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:08:58.038 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.039 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.040 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.041 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.042 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3954289788", 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3954289788", 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3954289788", 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3954289788", 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:08:58.043 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "76430717-38c9-43dd-9de4-8eee86d1bcbb", 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 14, 2026-02-26T03:08:58.044 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 27, 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 37, 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 31, 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.045 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.046 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.047 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:08:58.048 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 4113961987 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.049 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/4113961987", 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/4113961987", 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/4113961987", 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/4113961987", 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.050 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:08:00.941957+0000", 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:07:09.620937+0000", 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:08:01.211767+0000", 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:08:58.051 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:08:01.479225+0000", 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:07:26.887149+0000", 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.052 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "myprofile": { 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "crush-device-class": "", 2026-02-26T03:08:58.053 INFO:tasks.workunit.client.0.trial123.stdout: "crush-failure-domain": "osd", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "crush-num-failure-domains": "0", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "crush-osds-per-failure-domain": "0", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "crush-root": "default", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "jerasure-per-chunk-alignment": "false", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "m": "1", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van", 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "w": "8" 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:08:58.054 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/4004398083,v1:127.0.0.1:6805/4004398083]", 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:08:58.055 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:08:58.056 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:07:02.980022Z", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/4004398083,v1:127.0.0.1:6803/4004398083]", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/4004398083,v1:127.0.0.1:6809/4004398083]", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/4004398083,v1:127.0.0.1:6807/4004398083]", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:08:58.057 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3954289788,v1:127.0.0.1:6813/3954289788]", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:08:58.058 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:08:58.059 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:07:12.228010Z", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3954289788,v1:127.0.0.1:6811/3954289788]", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3954289788,v1:127.0.0.1:6817/3954289788]", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3954289788,v1:127.0.0.1:6815/3954289788]", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:08:58.060 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/4113961987,v1:127.0.0.1:6813/4113961987]", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:08:58.061 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:07:20.287986Z", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:08:58.062 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/4113961987,v1:127.0.0.1:6811/4113961987]", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/4113961987,v1:127.0.0.1:6817/4113961987]", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/4113961987,v1:127.0.0.1:6815/4113961987]", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.063 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 0, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [] 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 35 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 37 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 42, 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.064 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.065 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:08:58.066 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.067 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:08:58.068 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.069 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:08:58.070 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:08:58.071 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 1, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "test", 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "type": 3, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "op": "set_chooseleaf_tries", 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "num": 5 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "op": "set_choose_tries", 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: "num": 100 2026-02-26T03:08:58.072 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stderr:report 2543591148 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_indep", 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.073 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:08:58.074 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T03:07:00:316971+0000", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:08:58.075 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 1, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73720, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 101, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 1, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 303, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 101, 2026-02-26T03:08:58.076 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 101, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:08:58.077 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 18, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 18, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "up": 3, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 3, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:08:58.078 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83912, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2132, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488888, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036621312, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2183168, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 1543685, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:08:58.079 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 83912, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2132, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 10, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 81717, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314488888, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322036621312, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2183168, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 1543685, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:08:58.080 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 10389, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 83679083 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73720, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 101, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 1, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 303, 2026-02-26T03:08:58.081 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 101, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 101, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:08:58.082 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1241088, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 1241088, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 18, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 18, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "up": 3, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 3, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 150323855372 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 42949672977 2026-02-26T03:08:58.083 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 158913789963 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num": 1 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2147483647, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 1, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.084 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 4, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:08:18.792468+0000", 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "0": { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:08:58.085 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 56, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 256, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 365, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.0 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:08:58.086 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:08:58.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:08:58.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:08:58.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:08:58.127 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:08:58.127 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 0 --op log --pgid 1.0 2026-02-26T03:08:58.127 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:08:58.128 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:08:58.128 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-26T03:08:58.128 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:08:58.128 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/0 2026-02-26T03:08:58.128 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/0 --op log --pgid 1.0 2026-02-26T03:08:58.769 INFO:tasks.workunit.client.0.trial123.stderr:PG '1.0' not found 2026-02-26T03:08:59.314 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-26T03:08:59.314 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-26T03:08:59.314 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-26T03:08:59.315 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:08:59.316 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:08:59.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:08:59.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:08:59.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-26T03:08:59.319 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:08:59.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:08:59.338 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:59.334+0000 7f215610c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:59.339 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:59.338+0000 7f215610c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:59.340 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:08:59.338+0000 7f215610c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:08:59.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:08:59.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:08:59.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:08:59.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:08:59.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:08:59.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:08:59.519 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:08:59.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:08:59.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:08:59.520 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:08:59.712 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 35 up_thru 41 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6802/4004398083,v1:127.0.0.1:6803/4004398083] [v2:127.0.0.1:6804/4004398083,v1:127.0.0.1:6805/4004398083] exists,up 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.2 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:08:59.713 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:08:59.714 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:08:59.714 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:08:59.714 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:08:59.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:08:59.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 2 --op log --pgid 1.0 2026-02-26T03:08:59.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:08:59.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:08:59.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-26T03:08:59.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:08:59.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/2 2026-02-26T03:08:59.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/2 --op log --pgid 1.0 2026-02-26T03:09:00.467 INFO:tasks.workunit.client.0.trial123.stderr:PG '1.0' not found 2026-02-26T03:09:00.825 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:00.822+0000 7f215610c8c0 -1 Falling back to public interface 2026-02-26T03:09:01.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-26T03:09:01.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-26T03:09:01.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:09:01.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:09:01.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:09:01.007 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:09:01.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:09:01.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:09:01.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:09:01.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:09:01.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-26T03:09:01.009 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:09:01.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:09:01.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:09:01.011 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-26T03:09:01.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:09:01.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:09:01.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:09:01.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:09:01.027 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:01.022+0000 7f3c1de038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:01.028 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:01.026+0000 7f3c1de038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:01.029 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:01.026+0000 7f3c1de038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:01.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:09:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:09:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:09:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:09:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:09:01.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:09:01.204 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:09:01.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:09:01.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:09:01.205 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:09:01.411 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 37 up_thru 0 down_at 31 last_clean_interval [14,27) [v2:127.0.0.1:6810/4113961987,v1:127.0.0.1:6811/4113961987] [v2:127.0.0.1:6812/4113961987,v1:127.0.0.1:6813/4113961987] exists,up 76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:336: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 1 --op log --pgid 1.0 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-26T03:09:01.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op log --pgid 1.0 2026-02-26T03:09:02.100 INFO:tasks.workunit.client.0.trial123.stderr:PG '1.0' not found 2026-02-26T03:09:02.293 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:02.290+0000 7f3c1de038c0 -1 Falling back to public interface 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: wait_for_clean 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:09:02.653 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-26T03:09:02.655 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:09:02.655 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:09:02.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:09:02.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:09:02.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:09:02.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-26T03:09:02.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:09:02.716 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:09:02.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:09:02.717 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:09:02.717 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:09:02.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:09:02.916 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:09:02.916 INFO:tasks.workunit.client.0.trial123.stderr:2' 2026-02-26T03:09:02.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:09:02.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:09:02.916 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:09:03.892 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:03.890+0000 7f215610c8c0 -1 osd.0 42 log_to_monitors true 2026-02-26T03:09:05.342 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:05.338+0000 7f3c1de038c0 -1 osd.2 42 log_to_monitors true 2026-02-26T03:09:08.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528323 2026-02-26T03:09:08.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528323 2026-02-26T03:09:08.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528323' 2026-02-26T03:09:08.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:09:08.011 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:09:08.069 INFO:tasks.workunit.client.0.trial123.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-26T03:09:08.072 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-26T03:09:08.072 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-26T03:09:08.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-26T03:09:08.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:09:08.073 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:09:08.152 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397507 2026-02-26T03:09:08.152 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397507 2026-02-26T03:09:08.152 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528323 2-210453397507' 2026-02-26T03:09:08.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:09:08.153 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-193273528323 2026-02-26T03:09:08.153 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:09:08.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:09:08.155 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-193273528323 2026-02-26T03:09:08.155 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:09:08.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528323 2026-02-26T03:09:08.156 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 193273528323' 2026-02-26T03:09:08.157 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 193273528323 2026-02-26T03:09:08.157 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:09:08.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 193273528323 2026-02-26T03:09:08.360 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:09:09.361 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:09:09.361 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:09:09.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528323 -lt 193273528323 2026-02-26T03:09:09.566 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:09:09.567 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-210453397507 2026-02-26T03:09:09.567 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:09:09.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:09:09.569 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-210453397507 2026-02-26T03:09:09.569 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:09:09.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397507 2026-02-26T03:09:09.571 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 210453397507 2026-02-26T03:09:09.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 210453397507' 2026-02-26T03:09:09.571 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:09:09.776 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-26T03:09:09.777 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:09:09.777 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:09.777 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:10.049 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:10.253 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:10.254 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:10.254 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:10.254 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:10.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:10.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-26T03:09:10.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-26T03:09:10.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-26T03:09:10.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:10.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-26T03:09:10.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:10.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:10.621 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:10.621 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:10.621 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:10.621 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:10.622 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:10.622 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:10.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:10.820 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:10.821 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:10.821 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:11.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:11.085 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:11.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:11.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:11.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-26T03:09:11.357 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:11.358 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-26T03:09:11.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:11.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:11.558 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:11.558 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:11.558 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:11.558 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:11.558 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:11.558 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:11.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:11.761 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:11.761 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:11.762 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:12.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:12.028 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:12.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:12.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:12.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-26T03:09:12.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:12.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-26T03:09:12.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:12.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:12.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:12.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:12.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:12.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:12.698 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:12.698 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:12.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:12.901 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:12.901 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:12.901 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:13.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:13.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:13.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:13.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:13.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:13.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:13.168 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:13.168 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:13.168 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:13.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:13.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:13.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-26T03:09:13.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:13.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-26T03:09:14.237 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:14.237 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:14.237 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:14.238 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:14.238 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:14.238 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:14.238 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:14.238 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:14.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:14.437 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:14.437 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:14.437 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:14.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:14.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:14.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:14.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:14.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:14.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:14.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:14.699 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:14.699 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:14.957 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:14.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:14.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-26T03:09:14.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:14.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-26T03:09:16.558 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:16.558 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:16.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:16.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:16.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:16.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:16.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:16.560 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:16.755 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:16.756 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:16.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:16.757 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:17.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:17.013 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:17.013 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:17.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:17.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:17.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-26T03:09:17.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:17.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-26T03:09:20.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:20.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:20.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:20.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:20.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:20.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:20.478 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:20.479 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:20.672 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:20.673 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:20.674 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:20.674 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:20.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:20.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:20.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:20.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:20.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:20.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:20.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:20.930 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:20.930 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:21.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:21.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:21.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-26T03:09:21.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:21.190 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-26T03:09:27.590 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:27.591 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:27.592 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:27.592 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:27.592 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:27.592 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:27.592 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:27.592 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:27.786 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:27.787 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:27.787 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:27.787 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:28.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:28.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:28.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:28.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:28.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:28.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-26T03:09:28.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-26T03:09:28.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-26T03:09:35.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-26T03:09:35.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:09:35.612 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:09:35.612 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:09:35.612 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:09:35.612 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:09:35.612 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:09:35.612 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:09:35.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-26T03:09:35.809 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:09:35.810 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:09:35.810 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:09:36.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-26T03:09:36.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-26T03:09:36.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-26T03:09:36.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-26T03:09:36.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-26T03:09:36.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-26T03:09:36.582 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_fingerprint": "1031cb3a-2959-4a4f-83c3-ab3ff247c705", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "version": "19.2.3-687-g8d724284", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "commit": "8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "timestamp": "2026-02-26T03:09:36.582671+0000", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "tag": "", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "health": { 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "status": "HEALTH_WARN", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "checks": { 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "OSDMAP_FLAGS": { 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:09:36.583 INFO:tasks.workunit.client.0.trial123.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "count": 17 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [], 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "OSD_DOWN": { 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 osds down", 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.584 INFO:tasks.workunit.client.0.trial123.stdout: "message": "osd.1 (root=default,host=trial123) is down" 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "PG_DEGRADED": { 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.585 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "severity": "HEALTH_WARN", 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "summary": { 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "count": 1 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "detail": [ 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "message": "application not enabled on pool 'test'" 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-26T03:09:36.586 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "muted": false 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "mutes": [] 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_first_committed": 1, 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "monmap_last_committed": 1, 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "monmap": { 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:07:00.236386Z", 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:07:00.236386Z", 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release": 19, 2026-02-26T03:09:36.587 INFO:tasks.workunit.client.0.trial123.stdout: "min_mon_release_name": "squid", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "election_strategy": 1, 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "disallowed_leaders": "", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": false, 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "tiebreaker_mon": "", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "removed_ranks": "", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "features": { 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "persistent": [ 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "kraken", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "luminous", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "mimic", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap-prune", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "nautilus", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "octopus", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "pacific", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "elector-pinging", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "quincy", 2026-02-26T03:09:36.588 INFO:tasks.workunit.client.0.trial123.stdout: "reef", 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "squid" 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "optional": [] 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "mons": [ 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "rank": 0, 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "name": "a", 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115", 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 0 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.589 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:7115/0", 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "priority": 0, 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 0, 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "crush_location": "{}" 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "quorum": [ 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: 0 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap": { 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 50, 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "fsid": "8aab73df-2bf9-48b7-981f-ebd42fdc7c52", 2026-02-26T03:09:36.590 INFO:tasks.workunit.client.0.trial123.stdout: "created": "2026-02-26T03:07:00.317191+0000", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:09:05.724174+0000", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "last_up_change": "2026-02-26T03:09:05.621448+0000", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "last_in_change": "2026-02-26T03:08:13.198836+0000", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "flags_num": 5800384, 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "flags_set": [ 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "nodown", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "noin", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "noout", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "pglog_hardlimit", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snapdirs", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "recovery_deletes", 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: "sortbitwise" 2026-02-26T03:09:36.591 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "crush_version": 12, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "full_ratio": 0.99000000953674316, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_snapshot": "", 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "pool_max": 1, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "max_osd": 3, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "require_min_compat_client": "luminous", 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "min_compat_client": "jewel", 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "require_osd_release": "squid", 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "allow_crimson": false, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "pools": [ 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "pool_name": "test", 2026-02-26T03:09:36.592 INFO:tasks.workunit.client.0.trial123.stdout: "create_time": "2026-02-26T03:07:28.977995+0000", 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "flags": 16389, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "type": 3, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "size": 3, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "min_size": 2, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "crush_rule": 1, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_count": 0, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_target": 0, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_barrier": 0, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "is_stretch_pool": false, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "object_hash": 2, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "pg_autoscale_mode": "off", 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num": 1, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num": 1, 2026-02-26T03:09:36.593 INFO:tasks.workunit.client.0.trial123.stdout: "pg_placement_num_target": 1, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_target": 1, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "pg_num_pending": 1, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_pg_merge_meta": { 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "source_pgid": "0.0", 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "ready_epoch": 0, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_started": 0, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": 0, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "source_version": "0'0", 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "target_version": "0'0" 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_change": "28", 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend": "0", 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "last_force_op_resend_preluminous": "0", 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "auid": 0, 2026-02-26T03:09:36.594 INFO:tasks.workunit.client.0.trial123.stdout: "snap_mode": "pool", 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "snap_seq": 1, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "snap_epoch": 28, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "pool_snaps": [ 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "snapid": 1, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "stamp": "2026-02-26T03:07:50.332149+0000", 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "name": "snap1" 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps": "[]", 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_bytes": 0, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "quota_max_objects": 0, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "tiers": [], 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "tier_of": -1, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "read_tier": -1, 2026-02-26T03:09:36.595 INFO:tasks.workunit.client.0.trial123.stdout: "write_tier": -1, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "cache_mode": "none", 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_bytes": 0, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "target_max_objects": 0, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_flush_age": 0, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "cache_min_evict_age": 0, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profile": "myprofile", 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_params": { 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "type": "none" 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_period": 0, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_count": 0, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "use_gmt_hitset": true, 2026-02-26T03:09:36.596 INFO:tasks.workunit.client.0.trial123.stdout: "min_read_recency_for_promote": 0, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "min_write_recency_for_promote": 0, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_grade_decay_rate": 0, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "hit_set_search_last_n": 0, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "grade_table": [], 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "stripe_width": 8192, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "expected_num_objects": 0, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "fast_read": false, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "options": {}, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "application_metadata": {} 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "osds": [ 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1", 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:09:36.597 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 35, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 42, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 45, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 49, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 44, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.598 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6802", 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6803", 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6804", 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.599 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6805", 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6808", 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.600 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6809", 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6806", 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6807", 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 1090569946 2026-02-26T03:09:36.601 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6803/1090569946", 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6805/1090569946", 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6809/1090569946", 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6807/1090569946", 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "fa6691a7-52bf-4ea2-ac24-ed6cd404391d", 2026-02-26T03:09:36.602 INFO:tasks.workunit.client.0.trial123.stdout: "up": 0, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 0, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 0, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 10, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 23, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 30, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.603 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.604 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:09:36.605 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3954289788 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.606 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3954289788", 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3954289788", 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3954289788", 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3954289788", 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "exists" 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "uuid": "76430717-38c9-43dd-9de4-8eee86d1bcbb", 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "up": 1, 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "in": 1, 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 1, 2026-02-26T03:09:36.607 INFO:tasks.workunit.client.0.trial123.stdout: "primary_affinity": 1, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_begin": 37, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "last_clean_end": 42, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 49, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "up_thru": 0, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "down_at": 48, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "lost_at": 0, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "public_addrs": { 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6810", 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6811", 2026-02-26T03:09:36.608 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addrs": { 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6812", 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6813", 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.609 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addrs": { 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6816", 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6817", 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addrs": { 2026-02-26T03:09:36.610 INFO:tasks.workunit.client.0.trial123.stdout: "addrvec": [ 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v2", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6814", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "type": "v1", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "127.0.0.1:6815", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "nonce": 3649644372 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "public_addr": "127.0.0.1:6811/3649644372", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "cluster_addr": "127.0.0.1:6813/3649644372", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3649644372", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3649644372", 2026-02-26T03:09:36.611 INFO:tasks.workunit.client.0.trial123.stdout: "state": [ 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "exists", 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "up" 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "osd_xinfo": [ 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:09:04.047570+0000", 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:07:09.620937+0000", 2026-02-26T03:09:36.612 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:08:01.211767+0000", 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "down_stamp": "2026-02-26T03:09:05.520829+0000", 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_probability": 0, 2026-02-26T03:09:36.613 INFO:tasks.workunit.client.0.trial123.stdout: "laggy_interval": 0, 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "features": 4540701547738038271, 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "old_weight": 0, 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "last_purged_snaps_scrub": "2026-02-26T03:07:26.887149+0000", 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "dead_epoch": 0 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap": [], 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_items": [], 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "pg_upmap_primaries": [], 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "pg_temp": [], 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "primary_temp": [], 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "blocklist": {}, 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "range_blocklist": {}, 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "erasure_code_profiles": { 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "default": { 2026-02-26T03:09:36.614 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "m": "2", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van" 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "myprofile": { 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "crush-device-class": "", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "crush-failure-domain": "osd", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "crush-num-failure-domains": "0", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "crush-osds-per-failure-domain": "0", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "crush-root": "default", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "jerasure-per-chunk-alignment": "false", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "k": "2", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "m": "1", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "plugin": "jerasure", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "technique": "reed_sol_van", 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: "w": "8" 2026-02-26T03:09:36.615 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "removed_snaps_queue": [], 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "new_removed_snaps": [], 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "new_purged_snaps": [], 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "crush_node_flags": {}, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "device_class_flags": {}, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode": { 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_enabled": false, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_bucket_count": 0, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "degraded_stretch_mode": 0, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "recovering_stretch_mode": 0, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "stretch_mode_bucket": 0 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "osd_metadata": [ 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.616 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6804/1090569946,v1:127.0.0.1:6805/1090569946]", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-26T03:09:36.617 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:07:02.980022Z", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:09:36.618 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6802/1090569946,v1:127.0.0.1:6803/1090569946]", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/1090569946,v1:127.0.0.1:6809/1090569946]", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/1090569946,v1:127.0.0.1:6807/1090569946]", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:09:36.619 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/0", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3954289788,v1:127.0.0.1:6813/3954289788]", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:09:36.620 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:09:36.621 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:07:12.228010Z", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3954289788,v1:127.0.0.1:6811/3954289788]", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3954289788,v1:127.0.0.1:6817/3954289788]", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3954289788,v1:127.0.0.1:6815/3954289788]", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:09:36.622 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/1", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "arch": "x86_64", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "back_addr": "[v2:127.0.0.1:6812/3649644372,v1:127.0.0.1:6813/3649644372]", 2026-02-26T03:09:36.623 INFO:tasks.workunit.client.0.trial123.stdout: "back_iface": "", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs": "1", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_db": "0", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_dedicated_wal": "0", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluefs_single_shared_device": "1", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocation_from_file": "1", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_allocator": "hybrid", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_access_mode": "file", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_block_size": "4096", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_rotational": "0", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_size": "107374182400", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_support_discard": "1", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_bdev_type": "ssd", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "bluestore_min_alloc_size": "4096", 2026-02-26T03:09:36.624 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_release": "squid", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_short": "19.2.3-687-g8d724284", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "ceph_version_when_created": "ceph version 19.2.3-687-g8d724284 (8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5) squid (stable)", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "created_at": "2026-02-26T03:07:20.287986Z", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "default_device_class": "ssd", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD7341", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "devices": "nvme0n1", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "distro": "ubuntu", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "distro_version": "22.04", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "front_addr": "[v2:127.0.0.1:6810/3649644372,v1:127.0.0.1:6811/3649644372]", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "front_iface": "", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3649644372,v1:127.0.0.1:6817/3649644372]", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3649644372,v1:127.0.0.1:6815/3649644372]", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "hostname": "trial123", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "journal_rotational": "0", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-26T03:09:36.625 INFO:tasks.workunit.client.0.trial123.stdout: "mem_swap_kb": "8388604", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "mem_total_kb": "130961696", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_node": "0", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "objectstore_numa_nodes": "0", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "os": "Linux", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "osd_data": "td/divergent-priors/2", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "osd_objectstore": "bluestore", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "osdspec_affinity": "", 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "rotational": "0" 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_clean_epochs": { 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "min_last_epoch_clean": 24, 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "last_epoch_clean": { 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "per_pool": [ 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: "floor": 24 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.626 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "osd_epochs": [ 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 46 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 50 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_first_committed": 1, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "osdmap_last_committed": 50, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "crushmap": { 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "devices": [ 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.0", 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.627 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.1", 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd.2", 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "class": "ssd" 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "types": [ 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 0, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "name": "osd" 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "name": "host" 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 2, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: "name": "chassis" 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.628 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 3, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "name": "rack" 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 4, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "name": "row" 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 5, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pdu" 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 6, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "name": "pod" 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 7, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "name": "room" 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 8, 2026-02-26T03:09:36.629 INFO:tasks.workunit.client.0.trial123.stdout: "name": "datacenter" 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 9, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "name": "zone" 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 10, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "name": "region" 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "name": "root" 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "buckets": [ 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "id": -1, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default", 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:09:36.630 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "id": -2, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "name": "default~ssd", 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 11, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "root", 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.631 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "id": -3, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123", 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:09:36.632 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "id": -4, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "name": "trial123~ssd", 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "type_id": 1, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "type_name": "host", 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 19206, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "alg": "straw2", 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "hash": "rjenkins1", 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "items": [ 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "id": 0, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 0 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "id": 1, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 1 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "id": 2, 2026-02-26T03:09:36.633 INFO:tasks.workunit.client.0.trial123.stdout: "weight": 6402, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "pos": 2 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "rules": [ 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 0, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "replicated_rule", 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "type": 1, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_firstn", 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.634 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "rule_id": 1, 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "rule_name": "test", 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "type": 3, 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "steps": [ 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "op": "set_chooseleaf_tries", 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "num": 5 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "op": "set_choose_tries", 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "num": 100 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.635 INFO:tasks.workunit.client.0.trial123.stdout: "op": "take", 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stderr:report 3855758564 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "item": -1, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "item_name": "default" 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "op": "choose_indep", 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "num": 0, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "type": "osd" 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "op": "emit" 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: ] 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "tunables": { 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_tries": 0, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "choose_local_fallback_tries": 0, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "choose_total_tries": 50, 2026-02-26T03:09:36.636 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_descend_once": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_vary_r": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "chooseleaf_stable": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "msr_descents": 100, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "msr_collision_tries": 100, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "straw_calc_version": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "allowed_bucket_algs": 54, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "profile": "jewel", 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "optimal_tunables": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "legacy_tunables": 0, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "minimum_required_version": "jewel", 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables2": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "has_v2_rules": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables3": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "has_v3_rules": 0, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "has_v4_buckets": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "require_feature_tunables5": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "has_v5_rules": 0, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "has_msr_rules": 0 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "choose_args": {} 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "fsmap": { 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "btime": "2026-02-26T03:07:00:316971+0000", 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "default_fscid": -1, 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "compat": { 2026-02-26T03:09:36.637 INFO:tasks.workunit.client.0.trial123.stdout: "compat": {}, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "ro_compat": {}, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "incompat": { 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_1": "base v0.20", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_2": "client writeable ranges", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_3": "default file layouts on dirs", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_4": "dir inode in separate object", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_5": "mds uses versioned encoding", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_8": "no anchor table", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_9": "file layout v2", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_10": "snaprealm v2", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_11": "minor log segments", 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_12": "quiesce subvolumes" 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "feature_flags": { 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "enable_multiple": true, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "ever_enabled_multiple": true 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "standbys": [], 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "filesystems": [] 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_first_committed": 1, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "mdsmap_last_committed": 1, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "auth": { 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:09:36.638 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 5, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_secrets": 3 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_active": 1, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_unknown": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_osd": 3, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "pool_sum": { 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73720, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 101, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 1, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 303, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 101, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 101, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:09:36.639 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:09:36.640 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 18, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 18, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "up": 3, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 3, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 0 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum": { 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 85092, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2260, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 15, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 82736, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314487708, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322035412992, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2314240, 2026-02-26T03:09:36.641 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 1598553, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 16250, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 84721798 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "osd_sum_by_class": { 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "ssd": { 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "up_from": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 0, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "num_pgs": 3, 2026-02-26T03:09:36.642 INFO:tasks.workunit.client.0.trial123.stdout: "num_osds": 3, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_osds": 3, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "num_per_pool_omap_osds": 3, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "kb": 314572800, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used": 85092, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_data": 2260, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_omap": 15, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "kb_used_meta": 82736, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "kb_avail": 314487708, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "statfs": { 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "total": 322122547200, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "available": 322035412992, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 2314240, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 1598553, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 16250, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 84721798 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "hb_peers": [], 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "snap_trim_queue_len": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "num_snap_trimming": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "num_shards_repaired": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "op_queue_age_hist": { 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "histogram": [], 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "upper_bound": 1 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "perf_stat": { 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ms": 0, 2026-02-26T03:09:36.643 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ms": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "commit_latency_ns": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "apply_latency_ns": 0 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "alerts": [], 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "network_ping_times": [] 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "pool_stats": [ 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "poolid": 1, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg": 1, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "stat_sum": { 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes": 73720, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects": 101, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_clones": 1, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_object_copies": 303, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing_on_primary": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_missing": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_degraded": 101, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_misplaced": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_unfound": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_dirty": 101, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_whiteouts": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_read": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_read_kb": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_write": 101, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_write_kb": 101, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_scrub_errors": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_shallow_scrub_errors": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_deep_scrub_errors": 0, 2026-02-26T03:09:36.644 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_recovered": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_recovered": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_keys_recovered": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_omap": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_hit_set_archive": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_bytes_hit_set_archive": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_kb": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_kb": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_promote": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_high": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_flush_mode_low": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_some": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_evict_mode_full": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_pinned": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_legacy_snapsets": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_large_omap_objects": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_manifest": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_bytes": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_omap_keys": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "num_objects_repaired": 0 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "store_stats": { 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "total": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "available": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "internally_reserved": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "allocated": 1241088, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "data_stored": 1241088, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_allocated": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "data_compressed_original": 0, 2026-02-26T03:09:36.645 INFO:tasks.workunit.client.0.trial123.stdout: "omap_allocated": 0, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "internal_metadata": 0 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "log_size": 18, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "ondisk_log_size": 18, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "up": 3, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "acting": 3, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "num_store_stats": 3 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "osd_stats": [ 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 193273528329 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 42949672977 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "seq": 210453397512 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_state": [ 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "state": "active+undersized+degraded", 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "num": 1 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "num_pg_by_osd": [ 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.646 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2147483647, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 2, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 0, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 1, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 1, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "osd": 1, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_primary_pg": 0, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_acting_pg": 0, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "num_up_not_acting_pg": 0 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": [ 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "pool": 1, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "purged_snaps": {} 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: ], 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "servicemap": { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "epoch": 4, 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "modified": "2026-02-26T03:08:18.792468+0000", 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "services": { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "osd": { 2026-02-26T03:09:36.647 INFO:tasks.workunit.client.0.trial123.stdout: "daemons": { 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "summary": "", 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "0": { 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "2": { 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "start_epoch": 0, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "start_stamp": "0.000000", 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "gid": 0, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "metadata": {}, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "task_status": {} 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: }, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_first_committed": 1, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "mgrstat_last_committed": 75, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "logm_first_committed": 1, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "logm_last_committed": 334, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "paxos": { 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "first_committed": 1, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "last_committed": 470, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "last_pn": 0, 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: "accepted_pn": 0 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout: } 2026-02-26T03:09:36.648 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:09:36.649 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-26T03:09:36.649 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:341: TEST_divergent_ec: echo 'delay recovery' 2026-02-26T03:09:36.649 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery 2026-02-26T03:09:36.649 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:342: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:36.780 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:36.780 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 50:149 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 no scrub is scheduled 0 0 2026-02-26T03:09:36.781 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:36.781 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:36.781 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:36.789 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:09:36.790 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.0 2026-02-26T03:09:36.790 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-26T03:09:36.790 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-26T03:09:36.790 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:09:36.790 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:09:36.790 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:09:36.791 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.0.asok 2026-02-26T03:09:36.791 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-26T03:09:36.791 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.0.asok set_recovery_delay 100000 2026-02-26T03:09:36.848 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:09:36.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-26T03:09:36.854 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.2 2026-02-26T03:09:36.854 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-26T03:09:36.854 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-26T03:09:36.855 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:09:36.855 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:09:36.855 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:09:36.855 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.2.asok 2026-02-26T03:09:36.855 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-26T03:09:36.855 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.2.asok set_recovery_delay 100000 2026-02-26T03:09:36.909 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:09:36.915 INFO:tasks.workunit.client.0.trial123.stdout:revive divergent 1 2026-02-26T03:09:36.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:349: TEST_divergent_ec: echo 'revive divergent 1' 2026-02-26T03:09:36.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:350: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:37.115 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:37.115 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 50:149 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 no scrub is scheduled 0 0 2026-02-26T03:09:37.115 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:37.115 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:37.115 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:37.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:351: TEST_divergent_ec: ceph osd set noup 2026-02-26T03:09:37.445 INFO:tasks.workunit.client.0.trial123.stderr:noup is set 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:352: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:09:37.457 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:09:37.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:09:37.459 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:09:37.459 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:09:37.459 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:09:37.460 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:09:37.461 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:09:37.461 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:09:37.461 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:09:37.461 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:09:37.476 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:37.473+0000 7f5ddb56d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:37.478 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:37.473+0000 7f5ddb56d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:37.479 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:37.477+0000 7f5ddb56d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:37.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-26T03:09:37.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:353: TEST_divergent_ec: sleep 5 2026-02-26T03:09:38.701 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:38.697+0000 7f5ddb56d8c0 -1 Falling back to public interface 2026-02-26T03:09:41.676 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:41.673+0000 7f5ddb56d8c0 -1 osd.1 28 log_to_monitors true 2026-02-26T03:09:42.668 INFO:tasks.workunit.client.0.trial123.stdout:delay recovery divergent 2026-02-26T03:09:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:355: TEST_divergent_ec: echo 'delay recovery divergent' 2026-02-26T03:09:42.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:356: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:42.862 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:42.862 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 52:153 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:55:08.484745+0000 0 0 2026-02-26T03:09:42.863 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:42.863 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:42.863 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:42.872 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: get_asok_path osd.1 2026-02-26T03:09:42.872 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-26T03:09:42.872 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-26T03:09:42.873 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:09:42.873 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:09:42.873 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:09:42.873 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.18886/ceph-osd.1.asok 2026-02-26T03:09:42.873 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: CEPH_ARGS= 2026-02-26T03:09:42.873 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.18886/ceph-osd.1.asok set_recovery_delay 100000 2026-02-26T03:09:42.927 INFO:tasks.workunit.client.0.trial123.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-26T03:09:42.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:359: TEST_divergent_ec: ceph osd unset noup 2026-02-26T03:09:43.251 INFO:tasks.workunit.client.0.trial123.stderr:noup is unset 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:361: TEST_divergent_ec: wait_for_osd up 0 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:09:43.262 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:09:43.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:09:43.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:09:43.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:09:43.471 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 45 up_thru 49 down_at 44 last_clean_interval [35,42) [v2:127.0.0.1:6802/1090569946,v1:127.0.0.1:6803/1090569946] [v2:127.0.0.1:6804/1090569946,v1:127.0.0.1:6805/1090569946] exists,up 3ef70ac4-86c1-419a-8d1f-88cbdfe92fb1 2026-02-26T03:09:43.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:09:43.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:09:43.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:09:43.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:362: TEST_divergent_ec: wait_for_osd up 1 2026-02-26T03:09:43.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:09:43.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:09:43.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:09:43.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:09:43.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:09:43.473 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:09:43.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:09:43.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:09:43.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:09:43.673 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 30 last_clean_interval [10,28) [v2:127.0.0.1:6818/3902609982,v1:127.0.0.1:6819/3902609982] [v2:127.0.0.1:6820/3902609982,v1:127.0.0.1:6821/3902609982] exists,up fa6691a7-52bf-4ea2-ac24-ed6cd404391d 2026-02-26T03:09:43.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:363: TEST_divergent_ec: wait_for_osd up 2 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:09:43.675 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:09:43.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:09:43.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:09:43.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:09:43.874 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 49 up_thru 0 down_at 48 last_clean_interval [37,42) [v2:127.0.0.1:6810/3649644372,v1:127.0.0.1:6811/3649644372] [v2:127.0.0.1:6812/3649644372,v1:127.0.0.1:6813/3649644372] exists,up 76430717-38c9-43dd-9de4-8eee86d1bcbb 2026-02-26T03:09:43.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:09:43.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:09:43.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:09:43.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:365: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:44.069 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:44.069 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 52:153 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:55:08.484745+0000 0 0 2026-02-26T03:09:44.069 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:44.069 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:44.069 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:44.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:366: TEST_divergent_ec: echo 'wait for peering' 2026-02-26T03:09:44.078 INFO:tasks.workunit.client.0.trial123.stdout:wait for peering 2026-02-26T03:09:44.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:367: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:44.284 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:44.284 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 52:153 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:55:08.484745+0000 0 0 2026-02-26T03:09:44.284 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:44.284 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:44.284 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:44.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:368: TEST_divergent_ec: rados -p test put foo /etc/fstab 2026-02-26T03:09:44.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:370: TEST_divergent_ec: echo 'killing divergent 1' 2026-02-26T03:09:44.323 INFO:tasks.workunit.client.0.trial123.stdout:killing divergent 1 2026-02-26T03:09:44.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:371: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:44.511 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:44.511 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 52:153 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:55:08.484745+0000 0 0 2026-02-26T03:09:44.511 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:44.511 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:44.511 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:44.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:372: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.1 2026-02-26T03:09:44.520 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:09:44.520 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:09:44.520 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:09:44.520 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:09:44.520 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:09:44.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:09:44.628 INFO:tasks.workunit.client.0.trial123.stdout:reviving divergent 1 2026-02-26T03:09:44.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:374: TEST_divergent_ec: echo 'reviving divergent 1' 2026-02-26T03:09:44.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:375: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:09:44.818 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:09:44.818 INFO:tasks.workunit.client.0.trial123.stdout:1.0 101 0 101 0 0 73720 0 0 18 84 18 active+undersized+degraded 2026-02-26T03:09:05.732809+0000 42'102 52:153 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T10:55:08.484745+0000 0 0 2026-02-26T03:09:44.818 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:09:44.818 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:09:44.818 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:09:44.827 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:376: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-26T03:09:44.827 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-26T03:09:44.827 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-26T03:09:44.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:09:44.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:09:44.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:09:44.831 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:09:44.832 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:09:44.833 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:09:44.833 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-26T03:09:44.833 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:09:44.833 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:09:44.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8aab73df-2bf9-48b7-981f-ebd42fdc7c52 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.18886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:09:44.834 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-26T03:09:44.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:09:44.834 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:09:44.835 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:09:44.835 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:09:44.857 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:44.853+0000 7f8da9fb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:44.858 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:44.853+0000 7f8da9fb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:44.859 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:44.857+0000 7f8da9fb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:09:45.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:09:45.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:09:45.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:09:45.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:09:45.038 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:09:45.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:09:45.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:09:45.039 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:09:45.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:09:45.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:09:45.240 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 30 last_clean_interval [10,28) [v2:127.0.0.1:6818/3902609982,v1:127.0.0.1:6819/3902609982] [v2:127.0.0.1:6820/3902609982,v1:127.0.0.1:6821/3902609982] exists,up fa6691a7-52bf-4ea2-ac24-ed6cd404391d 2026-02-26T03:09:45.241 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:09:45.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:09:45.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:09:45.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:378: TEST_divergent_ec: sleep 20 2026-02-26T03:09:46.361 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:46.357+0000 7f8da9fb18c0 -1 Falling back to public interface 2026-02-26T03:09:49.355 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:09:49.353+0000 7f8da9fb18c0 -1 osd.1 57 log_to_monitors true 2026-02-26T03:10:05.242 INFO:tasks.workunit.client.0.trial123.stdout:allowing recovery 2026-02-26T03:10:05.242 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:380: TEST_divergent_ec: echo 'allowing recovery' 2026-02-26T03:10:05.243 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:381: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:10:05.427 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:10:05.427 INFO:tasks.workunit.client.0.trial123.stdout:1.0 102 0 0 0 0 74448 0 0 13 90 13 active+clean 2026-02-26T03:09:49.750226+0000 57'103 61:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T04:32:50.311465+0000 0 0 2026-02-26T03:10:05.427 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:10:05.428 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:10:05.428 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:10:05.435 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-26T03:10:05.435 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-26T03:10:05.520 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:10:05.527 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-26T03:10:05.527 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-26T03:10:05.606 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:10:05.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-26T03:10:05.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-26T03:10:05.693 INFO:tasks.workunit.client.0.trial123.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-26T03:10:05.700 INFO:tasks.workunit.client.0.trial123.stdout:reading divergent objects 2026-02-26T03:10:05.700 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:388: TEST_divergent_ec: echo 'reading divergent objects' 2026-02-26T03:10:05.701 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:389: TEST_divergent_ec: ceph pg dump pgs 2026-02-26T03:10:05.894 INFO:tasks.workunit.client.0.trial123.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-26T03:10:05.894 INFO:tasks.workunit.client.0.trial123.stdout:1.0 102 0 0 0 0 74448 0 0 13 90 13 active+clean 2026-02-26T03:09:49.750226+0000 57'103 61:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-26T03:07:29.069557+0000 0'0 2026-02-26T03:07:29.069557+0000 0 0 periodic scrub scheduled @ 2026-02-27T04:32:50.311465+0000 0 0 2026-02-26T03:10:05.894 INFO:tasks.workunit.client.0.trial123.stdout: 2026-02-26T03:10:05.895 INFO:tasks.workunit.client.0.trial123.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-26T03:10:05.895 INFO:tasks.workunit.client.0.trial123.stderr:dumped pgs 2026-02-26T03:10:05.903 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: expr 2 + 2 2026-02-26T03:10:05.905 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: seq 1 4 2026-02-26T03:10:05.905 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:10:05.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_1 td/divergent-priors/existing 2026-02-26T03:10:05.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:10:05.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_2 td/divergent-priors/existing 2026-02-26T03:10:05.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:10:05.962 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_3 td/divergent-priors/existing 2026-02-26T03:10:05.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-26T03:10:05.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_4 td/divergent-priors/existing 2026-02-26T03:10:06.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:394: TEST_divergent_ec: rm -f td/divergent-priors/existing 2026-02-26T03:10:06.011 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-26T03:10:06.012 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:10:06.016 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.493+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:1 entries: 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-26T03:07:45.664185+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:10:06.016 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.493+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: keeping 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-26T03:07:45.664185+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-26T03:10:06.016 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.493+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:10:06.017 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.493+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version: 27'99 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-26T03:10:06.017 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.493+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 has no more recent entries in log 2026-02-26T03:10:06.017 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.493+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version or op type indicates creation, deleting 2026-02-26T03:10:06.017 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:head entries: 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4587.0:1 2026-02-26T03:07:50.407029+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-26T03:10:06.017 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: keeping 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4587.0:1 2026-02-26T03:07:50.407029+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-26T03:10:06.017 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:10:06.018 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head prior_version: 27'99 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-26T03:10:06.018 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head has no more recent entries in log 2026-02-26T03:10:06.018 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:10:06.018 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head olog_can_rollback_to: 28'104 2026-02-26T03:10:06.018 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot rollback 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4587.0:1 2026-02-26T03:07:50.407029+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-26T03:10:06.018 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot roll back, removing and adding to missing 2026-02-26T03:10:06.019 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4578.0:1 2026-02-26T03:07:49.335975+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false,27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4583.0:1 2026-02-26T03:07:50.331728+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:10:06.019 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: keeping 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4578.0:1 2026-02-26T03:07:49.335975+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:10:06.020 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 20 _merge_object_divergent_entries: keeping 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4583.0:1 2026-02-26T03:07:50.331728+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:10:06.020 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-26T03:10:06.020 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 27'100 first_divergent_update: 27'101 last_divergent_update: 27'102 2026-02-26T03:10:06.020 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-26T03:10:06.020 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-26T03:10:06.020 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 28'104 2026-02-26T03:10:06.021 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4583.0:1 2026-02-26T03:07:50.331728+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-26T03:10:06.021 INFO:tasks.workunit.client.0.trial123.stdout:td/divergent-priors/osd.1.log:2026-02-26T03:09:43.497+0000 7f5dbd4f3640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-26T03:10:06.022 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-26T03:10:06.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: grep -q '_merge_object_divergent_entries.*rolled back' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-26T03:10:06.024 INFO:tasks.workunit.client.0.trial123.stdout:success 2026-02-26T03:10:06.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:410: TEST_divergent_ec: echo success 2026-02-26T03:10:06.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:412: TEST_divergent_ec: delete_pool test 2026-02-26T03:10:06.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-26T03:10:06.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-26T03:10:06.280 INFO:tasks.workunit.client.0.trial123.stderr:pool 'test' does not exist 2026-02-26T03:10:06.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:413: TEST_divergent_ec: kill_daemons td/divergent-priors 2026-02-26T03:10:06.291 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:10:06.291 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:10:06.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:10:06.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:10:06.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:10:14.608 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:10:14.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-26T03:10:14.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:10:14.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:10:14.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:10:14.609 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:10:14.609 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:10:14.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:10:14.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:10:14.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:10:14.617 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:10:14.618 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:10:14.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:10:14.619 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:10:14.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:10:14.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:10:14.621 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:10:14.622 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:10:14.622 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:10:14.622 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:10:14.623 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:10:14.624 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:10:14.625 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:10:14.625 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:10:14.648 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:10:14.648 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.648 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:10:14.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/divergent-priors 0 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:10:14.650 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:10:14.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:10:14.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:10:14.651 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:10:14.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:10:14.654 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:10:14.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:10:14.655 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:10:14.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:10:14.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:10:14.656 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:10:14.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:10:14.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:10:14.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:10:14.658 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:10:14.659 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:10:14.660 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-26T03:10:14.660 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-26T03:10:14.662 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:10:14.662 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.662 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.18886 2026-02-26T03:10:14.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.18886 2026-02-26T03:10:14.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:10:14.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:10:14.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-26T03:10:14.664 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-26T03:10:14.664 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T03:10:14.671 INFO:tasks.workunit:Running workunit osd/ec-error-rollforward.sh... 2026-02-26T03:10:14.671 DEBUG:teuthology.orchestra.run.trial123:workunit test osd/ec-error-rollforward.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh 2026-02-26T03:10:14.723 INFO:tasks.workunit.client.0.trial123.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/ec-error-rollforward 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:6: run: local dir=td/ec-error-rollforward 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:7: run: shift 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: export CEPH_MON=127.0.0.1:7132 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: CEPH_MON=127.0.0.1:7132 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:11: run: export CEPH_ARGS 2026-02-26T03:10:14.727 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: uuidgen 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: CEPH_ARGS+='--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none ' 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:13: run: CEPH_ARGS+='--mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: export margin=10 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: margin=10 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: export objects=200 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: objects=200 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: export poolname=test 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: poolname=test 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: set 2026-02-26T03:10:14.728 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: local funcs=TEST_ec_error_rollforward 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:19: run: for func in $funcs 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:20: run: setup td/ec-error-rollforward 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/ec-error-rollforward 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/ec-error-rollforward 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:10:14.730 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:10:14.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:10:14.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:10:14.731 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:10:14.733 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:10:14.733 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:10:14.734 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:10:14.734 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:10:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:10:14.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:10:14.736 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:10:14.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:10:14.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:10:14.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:10:14.737 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:10:14.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:10:14.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:10:14.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-26T03:10:14.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:10:14.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:14.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.79512 2026-02-26T03:10:14.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:10:14.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:10:14.742 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/ec-error-rollforward 2026-02-26T03:10:14.744 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:10:14.744 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.744 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:14.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.79512 2026-02-26T03:10:14.746 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/ec-error-rollforward 1' TERM HUP INT 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:21: run: TEST_ec_error_rollforward td/ec-error-rollforward 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:27: TEST_ec_error_rollforward: local dir=td/ec-error-rollforward 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:28: TEST_ec_error_rollforward: run_mon td/ec-error-rollforward a 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/ec-error-rollforward 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/ec-error-rollforward/a 2026-02-26T03:10:14.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/ec-error-rollforward/a --run-dir=td/ec-error-rollforward 2026-02-26T03:10:14.770 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:10:14.770 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:14.770 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:14.771 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:14.771 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.771 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:14.771 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:14.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ec-error-rollforward/a '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --mon-cluster-log-file=td/ec-error-rollforward/log --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:10:14.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:10:14.803 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:10:14.803 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:10:14.803 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:10:14.803 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.804 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:14.805 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.79512/ceph-mon.a.asok 2026-02-26T03:10:14.805 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:10:14.805 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.79512/ceph-mon.a.asok config get fsid 2026-02-26T03:10:14.860 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:14.861 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:14.862 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.79512/ceph-mon.a.asok 2026-02-26T03:10:14.862 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:10:14.862 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.79512/ceph-mon.a.asok config get mon_host 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:29: TEST_ec_error_rollforward: run_mgr td/ec-error-rollforward x 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/ec-error-rollforward 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/ec-error-rollforward/x 2026-02-26T03:10:14.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:10:15.020 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:10:15.020 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:15.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:15.021 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:15.021 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:15.021 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:15.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:15.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:10:15.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ec-error-rollforward/x '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:10:15.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:30: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 0 2026-02-26T03:10:15.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:15.041 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/0 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:10:15.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:10:15.043 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/0 2026-02-26T03:10:15.044 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:10:15.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2f97101f-bd56-4709-b465-290ccb852358 2026-02-26T03:10:15.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2f97101f-bd56-4709-b465-290ccb852358' 2026-02-26T03:10:15.045 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 2f97101f-bd56-4709-b465-290ccb852358 2026-02-26T03:10:15.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:10:15.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCXuZ9pym4lAxAAHVEVwDHIHIxCxDWVwHosPg== 2026-02-26T03:10:15.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCXuZ9pym4lAxAAHVEVwDHIHIxCxDWVwHosPg=="}' 2026-02-26T03:10:15.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2f97101f-bd56-4709-b465-290ccb852358 -i td/ec-error-rollforward/0/new.json 2026-02-26T03:10:15.165 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:15.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/0/new.json 2026-02-26T03:10:15.175 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCXuZ9pym4lAxAAHVEVwDHIHIxCxDWVwHosPg== --osd-uuid 2f97101f-bd56-4709-b465-290ccb852358 2026-02-26T03:10:15.190 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:15.189+0000 7f8883d8f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:15.192 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:15.189+0000 7f8883d8f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:15.193 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:15.189+0000 7f8883d8f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:15.193 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:15.189+0000 7f8883d8f8c0 -1 bdev(0x55ea16c35c00 td/ec-error-rollforward/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:10:15.194 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:15.189+0000 7f8883d8f8c0 -1 bluestore(td/ec-error-rollforward/0) _read_fsid unparsable uuid 2026-02-26T03:10:18.031 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/0/keyring 2026-02-26T03:10:18.031 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:10:18.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:10:18.032 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:10:18.032 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:10:18.297 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:10:18.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:10:18.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:18.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:10:18.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:10:18.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:10:18.319 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:18.317+0000 7fc40080b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:18.321 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:18.317+0000 7fc40080b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:18.322 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:18.317+0000 7fc40080b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:18.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:10:18.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:18.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:10:18.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:18.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:18.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:18.502 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:18.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:18.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:18.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:18.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:19.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:19.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:19.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:10:19.704 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:10:19.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:19.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:19.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:20.056 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:20.053+0000 7fc40080b8c0 -1 Falling back to public interface 2026-02-26T03:10:20.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:20.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:20.897 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:10:20.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:10:20.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:20.898 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:21.092 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:22.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:22.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:22.094 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:10:22.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:10:22.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:22.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:22.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:23.014 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:23.013+0000 7fc40080b8c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:10:23.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:23.297 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:23.297 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:10:23.297 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:10:23.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:23.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:23.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:24.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:24.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:24.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:10:24.500 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:10:24.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:24.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:24.695 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3736262010,v1:127.0.0.1:6803/3736262010] [v2:127.0.0.1:6804/3736262010,v1:127.0.0.1:6805/3736262010] exists,up 2f97101f-bd56-4709-b465-290ccb852358 2026-02-26T03:10:24.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:24.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:24.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:24.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:31: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 1 2026-02-26T03:10:24.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:24.703 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/1 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:24.704 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:24.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-26T03:10:24.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-26T03:10:24.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:24.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:10:24.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:24.705 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:10:24.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:24.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:24.706 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:24.706 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:24.706 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:24.706 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:24.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:24.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:24.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:24.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:24.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:10:24.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:10:24.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/1 2026-02-26T03:10:24.709 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:10:24.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c39868e1-10a9-4e93-899e-ee117fc9151e 2026-02-26T03:10:24.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 c39868e1-10a9-4e93-899e-ee117fc9151e' 2026-02-26T03:10:24.710 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 c39868e1-10a9-4e93-899e-ee117fc9151e 2026-02-26T03:10:24.710 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:10:24.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCguZ9pw+p5KhAAL8LHqMwApbNnUQ0Hc67LUg== 2026-02-26T03:10:24.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCguZ9pw+p5KhAAL8LHqMwApbNnUQ0Hc67LUg=="}' 2026-02-26T03:10:24.713 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c39868e1-10a9-4e93-899e-ee117fc9151e -i td/ec-error-rollforward/1/new.json 2026-02-26T03:10:24.917 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:10:24.926 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/1/new.json 2026-02-26T03:10:24.927 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCguZ9pw+p5KhAAL8LHqMwApbNnUQ0Hc67LUg== --osd-uuid c39868e1-10a9-4e93-899e-ee117fc9151e 2026-02-26T03:10:24.942 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:24.941+0000 7f0d46d648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:24.944 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:24.941+0000 7f0d46d648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:24.945 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:24.941+0000 7f0d46d648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:24.946 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:24.941+0000 7f0d46d648c0 -1 bdev(0x55ab0037fc00 td/ec-error-rollforward/1/block) open stat got: (1) Operation not permitted 2026-02-26T03:10:24.946 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:24.941+0000 7f0d46d648c0 -1 bluestore(td/ec-error-rollforward/1) _read_fsid unparsable uuid 2026-02-26T03:10:27.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/1/keyring 2026-02-26T03:10:27.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:10:27.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T03:10:27.236 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T03:10:27.237 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:10:27.498 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:10:27.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T03:10:27.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:27.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:10:27.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:10:27.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:10:27.520 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:27.516+0000 7f15c939c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:27.522 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:27.516+0000 7f15c939c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:27.523 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:27.520+0000 7f15c939c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:27.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T03:10:27.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:27.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:27.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:10:27.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:28.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:28.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:28.897 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:10:28.898 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:10:28.898 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:28.898 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:10:29.020 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:29.016+0000 7f15c939c8c0 -1 Falling back to public interface 2026-02-26T03:10:29.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:30.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:30.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:30.090 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:10:30.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:10:30.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:30.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:10:30.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:31.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:31.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:31.285 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:10:31.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:10:31.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:31.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:10:31.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:32.003 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:32.000+0000 7f15c939c8c0 -1 osd.1 0 log_to_monitors true 2026-02-26T03:10:32.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:32.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:32.485 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:10:32.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:10:32.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:32.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:10:32.677 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2097923457,v1:127.0.0.1:6811/2097923457] [v2:127.0.0.1:6812/2097923457,v1:127.0.0.1:6813/2097923457] exists,up c39868e1-10a9-4e93-899e-ee117fc9151e 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:32: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 2 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:10:32.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/2 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:32.681 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:32.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:32.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:32.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:10:32.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:10:32.683 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/2 2026-02-26T03:10:32.683 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:10:32.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=99ac85d6-0af7-4400-bf4d-a3b351f9ede4 2026-02-26T03:10:32.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 99ac85d6-0af7-4400-bf4d-a3b351f9ede4' 2026-02-26T03:10:32.684 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 99ac85d6-0af7-4400-bf4d-a3b351f9ede4 2026-02-26T03:10:32.684 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:10:32.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCouZ9pQxtQKRAAWwjyqtMrkIzK4jDpsyE37A== 2026-02-26T03:10:32.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCouZ9pQxtQKRAAWwjyqtMrkIzK4jDpsyE37A=="}' 2026-02-26T03:10:32.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 99ac85d6-0af7-4400-bf4d-a3b351f9ede4 -i td/ec-error-rollforward/2/new.json 2026-02-26T03:10:32.901 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:10:32.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/2/new.json 2026-02-26T03:10:32.911 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCouZ9pQxtQKRAAWwjyqtMrkIzK4jDpsyE37A== --osd-uuid 99ac85d6-0af7-4400-bf4d-a3b351f9ede4 2026-02-26T03:10:32.926 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:32.924+0000 7f427b8478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:32.928 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:32.924+0000 7f427b8478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:32.929 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:32.924+0000 7f427b8478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:32.930 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:32.924+0000 7f427b8478c0 -1 bdev(0x5608915e7c00 td/ec-error-rollforward/2/block) open stat got: (1) Operation not permitted 2026-02-26T03:10:32.930 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:32.924+0000 7f427b8478c0 -1 bluestore(td/ec-error-rollforward/2) _read_fsid unparsable uuid 2026-02-26T03:10:35.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/2/keyring 2026-02-26T03:10:35.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:10:35.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T03:10:35.212 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T03:10:35.213 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:10:35.488 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:10:35.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T03:10:35.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:35.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:10:35.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:10:35.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:10:35.509 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:35.504+0000 7f73b0d8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:35.510 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:35.508+0000 7f73b0d8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:35.511 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:35.508+0000 7f73b0d8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:35.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T03:10:35.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:35.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:10:35.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:35.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:35.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:35.681 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:35.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:35.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:35.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:35.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:36.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:36.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:36.881 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:10:36.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:10:36.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:36.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:37.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:37.256 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:37.252+0000 7f73b0d8a8c0 -1 Falling back to public interface 2026-02-26T03:10:38.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:38.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:38.086 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:10:38.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:10:38.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:38.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:38.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:39.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:39.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:39.286 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:10:39.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:10:39.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:39.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:39.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:40.297 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:40.292+0000 7f73b0d8a8c0 -1 osd.2 0 log_to_monitors true 2026-02-26T03:10:40.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:40.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:40.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:10:40.484 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:10:40.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:40.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:40.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:41.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:41.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:41.687 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:10:41.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:10:41.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:41.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:41.884 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1345389540,v1:127.0.0.1:6819/1345389540] [v2:127.0.0.1:6820/1345389540,v1:127.0.0.1:6821/1345389540] exists,up 99ac85d6-0af7-4400-bf4d-a3b351f9ede4 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:33: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 3 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:10:41.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/3 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:41.888 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:10:41.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/3 2026-02-26T03:10:41.890 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:10:41.890 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=31bb035f-f565-4ddb-adfe-653d8bf1abb4 2026-02-26T03:10:41.890 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 31bb035f-f565-4ddb-adfe-653d8bf1abb4' 2026-02-26T03:10:41.890 INFO:tasks.workunit.client.0.trial123.stdout:add osd3 31bb035f-f565-4ddb-adfe-653d8bf1abb4 2026-02-26T03:10:41.891 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:10:41.901 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCxuZ9pX0S3NRAA4uVKcOCq+3kGEQzAvHPsBA== 2026-02-26T03:10:41.901 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCxuZ9pX0S3NRAA4uVKcOCq+3kGEQzAvHPsBA=="}' 2026-02-26T03:10:41.901 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 31bb035f-f565-4ddb-adfe-653d8bf1abb4 -i td/ec-error-rollforward/3/new.json 2026-02-26T03:10:42.105 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:10:42.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/3/new.json 2026-02-26T03:10:42.115 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCxuZ9pX0S3NRAA4uVKcOCq+3kGEQzAvHPsBA== --osd-uuid 31bb035f-f565-4ddb-adfe-653d8bf1abb4 2026-02-26T03:10:42.130 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:42.128+0000 7f0ee64478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:42.131 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:42.128+0000 7f0ee64478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:42.133 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:42.128+0000 7f0ee64478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:42.133 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:42.128+0000 7f0ee64478c0 -1 bdev(0x55cf9206fc00 td/ec-error-rollforward/3/block) open stat got: (1) Operation not permitted 2026-02-26T03:10:42.133 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:42.128+0000 7f0ee64478c0 -1 bluestore(td/ec-error-rollforward/3) _read_fsid unparsable uuid 2026-02-26T03:10:44.443 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/3/keyring 2026-02-26T03:10:44.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:10:44.444 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-26T03:10:44.445 INFO:tasks.workunit.client.0.trial123.stdout:adding osd3 key to auth repository 2026-02-26T03:10:44.445 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:10:44.709 INFO:tasks.workunit.client.0.trial123.stdout:start osd.3 2026-02-26T03:10:44.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-26T03:10:44.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:44.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:10:44.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:10:44.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:10:44.724 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:44.720+0000 7ff8668ca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:44.725 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:44.720+0000 7ff8668ca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:44.726 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:44.724+0000 7ff8668ca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:44.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-26T03:10:44.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:44.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-26T03:10:44.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:44.905 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:44.905 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:44.905 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:44.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:44.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:44.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:45.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:46.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:46.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:46.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:10:46.103 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:10:46.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:46.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:46.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:46.448 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:46.444+0000 7ff8668ca8c0 -1 Falling back to public interface 2026-02-26T03:10:47.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:47.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:47.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:10:47.305 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:10:47.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:47.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:47.505 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:48.506 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:48.506 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:48.507 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:10:48.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:10:48.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:48.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:48.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:49.443 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:49.440+0000 7ff8668ca8c0 -1 osd.3 0 log_to_monitors true 2026-02-26T03:10:49.711 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:49.711 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:49.711 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:10:49.712 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:10:49.712 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:49.712 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:49.909 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:10:50.911 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:10:50.911 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:50.911 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:10:50.912 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:10:50.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:50.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:51.103 INFO:tasks.workunit.client.0.trial123.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2742057571,v1:127.0.0.1:6827/2742057571] [v2:127.0.0.1:6828/2742057571,v1:127.0.0.1:6829/2742057571] exists,up 31bb035f-f565-4ddb-adfe-653d8bf1abb4 2026-02-26T03:10:51.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:51.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:51.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:51.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:35: TEST_ec_error_rollforward: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-26T03:10:51.354 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:36: TEST_ec_error_rollforward: ceph osd pool create ec 1 1 erasure ec-profile 2026-02-26T03:10:51.702 INFO:tasks.workunit.client.0.trial123.stderr:pool 'ec' already exists 2026-02-26T03:10:51.711 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:38: TEST_ec_error_rollforward: rados -p ec put foo /etc/passwd 2026-02-26T03:10:52.965 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: cat td/ec-error-rollforward/osd.2.pid 2026-02-26T03:10:52.966 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: kill -STOP 84892 2026-02-26T03:10:52.966 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:43: TEST_ec_error_rollforward: pids=87326 2026-02-26T03:10:52.966 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:42: TEST_ec_error_rollforward: rados -p ec rm foo 2026-02-26T03:10:52.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:44: TEST_ec_error_rollforward: sleep 1 2026-02-26T03:10:53.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:46: TEST_ec_error_rollforward: pids+=' 87347' 2026-02-26T03:10:53.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:45: TEST_ec_error_rollforward: rados -p ec rm a 2026-02-26T03:10:53.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:48: TEST_ec_error_rollforward: pids+=' 87348' 2026-02-26T03:10:53.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:47: TEST_ec_error_rollforward: rados -p ec rm b 2026-02-26T03:10:53.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:50: TEST_ec_error_rollforward: pids+=' 87349' 2026-02-26T03:10:53.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:51: TEST_ec_error_rollforward: sleep 1 2026-02-26T03:10:53.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:49: TEST_ec_error_rollforward: rados -p ec rm c 2026-02-26T03:10:54.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:54: TEST_ec_error_rollforward: kill_daemons td/ec-error-rollforward KILL osd 2026-02-26T03:10:54.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:10:54.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:10:54.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:10:54.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:10:54.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:10:55.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:10:55.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:55: TEST_ec_error_rollforward: kill 87326 87347 87348 87349 2026-02-26T03:10:55.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:56: TEST_ec_error_rollforward: wait 2026-02-26T03:10:55.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:58: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 0 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/0 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:55.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:55.288 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:55.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:55.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:55.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:55.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:55.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:10:55.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:10:55.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/0 2026-02-26T03:10:55.292 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/0/whoami 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:10:55.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:10:55.312 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:55.308+0000 7ff08c49a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:55.314 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:55.308+0000 7ff08c49a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:55.315 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:55.312+0000 7ff08c49a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:55.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:10:55.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:55.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:10:55.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:55.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:55.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:55.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:55.483 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:55.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:55.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:10:55.684 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3736262010,v1:127.0.0.1:6803/3736262010] [v2:127.0.0.1:6804/3736262010,v1:127.0.0.1:6805/3736262010] exists,up 2f97101f-bd56-4709-b465-290ccb852358 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:59: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 1 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/1 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:55.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:10:55.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/1 2026-02-26T03:10:55.687 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:10:55.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:10:55.687 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:55.687 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/1/whoami 2026-02-26T03:10:55.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:10:55.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:10:55.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:10:55.689 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:10:55.703 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:55.700+0000 7f72af9038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:55.705 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:55.700+0000 7f72af9038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:55.705 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:55.700+0000 7f72af9038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:55.885 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:10:55.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:55.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:10:55.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:55.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:55.886 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:55.886 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:55.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:55.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:55.887 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2097923457,v1:127.0.0.1:6811/2097923457] [v2:127.0.0.1:6812/2097923457,v1:127.0.0.1:6813/2097923457] exists,up c39868e1-10a9-4e93-899e-ee117fc9151e 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:60: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 2 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/2 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-26T03:10:56.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:56.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:10:56.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/2 2026-02-26T03:10:56.087 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:10:56.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:10:56.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:56.087 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/2/whoami 2026-02-26T03:10:56.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:10:56.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:10:56.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:10:56.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:10:56.108 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:56.104+0000 7fa1e6fd68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:56.110 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:56.108+0000 7fa1e6fd68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:56.111 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:56.108+0000 7fa1e6fd68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:56.292 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:10:56.490 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1345389540,v1:127.0.0.1:6819/1345389540] [v2:127.0.0.1:6820/1345389540,v1:127.0.0.1:6821/1345389540] exists,up 99ac85d6-0af7-4400-bf4d-a3b351f9ede4 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:61: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 3 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-26T03:10:56.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/3 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-26T03:10:56.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:10:56.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:10:56.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:10:56.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/3 2026-02-26T03:10:56.498 INFO:tasks.workunit.client.0.trial123.stdout:start osd.3 2026-02-26T03:10:56.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-26T03:10:56.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=48084460-0089-44a6-8da8-c169186f28cc --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79512/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:10:56.500 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/3/whoami 2026-02-26T03:10:56.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-26T03:10:56.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:10:56.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:10:56.501 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:10:56.510 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:56.508+0000 7f5a5a55f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:56.512 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:56.508+0000 7f5a5a55f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:56.513 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:56.508+0000 7f5a5a55f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:10:56.691 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:10:56.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:10:56.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:10:56.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:10:56.894 INFO:tasks.workunit.client.0.trial123.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2742057571,v1:127.0.0.1:6827/2742057571] [v2:127.0.0.1:6828/2742057571,v1:127.0.0.1:6829/2742057571] exists,up 31bb035f-f565-4ddb-adfe-653d8bf1abb4 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:63: TEST_ec_error_rollforward: wait_for_clean 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:10:56.896 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:10:56.897 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:10:56.897 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:10:56.897 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:10:56.897 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:10:56.897 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:10:56.986 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:10:56.987 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:10:56.987 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:10:56.987 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:10:56.987 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:10:56.987 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:10:57.108 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:57.104+0000 7ff08c49a8c0 -1 Falling back to public interface 2026-02-26T03:10:57.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:10:57.188 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:10:57.188 INFO:tasks.workunit.client.0.trial123.stderr:2 2026-02-26T03:10:57.188 INFO:tasks.workunit.client.0.trial123.stderr:3' 2026-02-26T03:10:57.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:10:57.189 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:10:57.189 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:10:57.279 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:57.276+0000 7f72af9038c0 -1 Falling back to public interface 2026-02-26T03:10:57.376 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:57.372+0000 7fa1e6fd68c0 -1 Falling back to public interface 2026-02-26T03:10:58.088 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:10:58.084+0000 7f5a5a55f8c0 -1 Falling back to public interface 2026-02-26T03:11:00.228 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:00.224+0000 7ff08c49a8c0 -1 osd.0 23 log_to_monitors true 2026-02-26T03:11:00.382 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:00.376+0000 7f72af9038c0 -1 osd.1 23 log_to_monitors true 2026-02-26T03:11:00.681 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:00.676+0000 7fa1e6fd68c0 -1 osd.2 22 log_to_monitors true 2026-02-26T03:11:01.218 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:01.212+0000 7f5a5a55f8c0 -1 osd.3 23 log_to_monitors true 2026-02-26T03:11:02.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-26T03:11:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-26T03:11:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699' 2026-02-26T03:11:02.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:11:02.275 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:11:02.350 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084291 2026-02-26T03:11:02.350 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084291 2026-02-26T03:11:02.351 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699 1-120259084291' 2026-02-26T03:11:02.351 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:11:02.351 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:11:02.427 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986179 2026-02-26T03:11:02.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986179 2026-02-26T03:11:02.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699 1-120259084291 2-133143986179' 2026-02-26T03:11:02.428 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:11:02.428 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-26T03:11:02.505 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855363 2026-02-26T03:11:02.506 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855363 2026-02-26T03:11:02.506 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699 1-120259084291 2-133143986179 3-150323855363' 2026-02-26T03:11:02.506 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:11:02.506 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149699 2026-02-26T03:11:02.507 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:11:02.507 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:11:02.508 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149699 2026-02-26T03:11:02.508 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:11:02.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-26T03:11:02.510 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 111669149699 2026-02-26T03:11:02.510 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149699' 2026-02-26T03:11:02.510 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:11:02.719 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 111669149699 2026-02-26T03:11:02.719 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:11:03.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:11:03.720 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:11:03.917 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-26T03:11:03.917 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:11:03.918 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084291 2026-02-26T03:11:03.918 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:11:03.919 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:11:03.920 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084291 2026-02-26T03:11:03.920 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:11:03.921 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084291 2026-02-26T03:11:03.922 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084291' 2026-02-26T03:11:03.922 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 120259084291 2026-02-26T03:11:03.922 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:11:04.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084291 -lt 120259084291 2026-02-26T03:11:04.134 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:11:04.134 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-133143986179 2026-02-26T03:11:04.134 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:11:04.135 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:11:04.137 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-133143986179 2026-02-26T03:11:04.137 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:11:04.138 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986179 2026-02-26T03:11:04.139 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 133143986179' 2026-02-26T03:11:04.139 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 133143986179 2026-02-26T03:11:04.139 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:11:04.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986179 -lt 133143986179 2026-02-26T03:11:04.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:11:04.340 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-150323855363 2026-02-26T03:11:04.341 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:11:04.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-26T03:11:04.343 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-150323855363 2026-02-26T03:11:04.343 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:11:04.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855363 2026-02-26T03:11:04.344 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 150323855363' 2026-02-26T03:11:04.344 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.3 seq 150323855363 2026-02-26T03:11:04.344 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-26T03:11:04.553 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-26T03:11:04.553 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:11:04.554 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:11:04.554 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:11:04.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-26T03:11:04.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:11:04.822 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:11:04.822 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:11:04.822 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:11:04.823 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:11:04.823 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:11:04.823 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:11:05.016 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-26T03:11:05.016 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:11:05.017 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:11:05.017 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:11:05.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-26T03:11:05.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:11:05.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:11:05.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:22: run: teardown td/ec-error-rollforward 2026-02-26T03:11:05.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-26T03:11:05.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:11:05.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-26T03:11:05.284 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:11:05.284 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:11:05.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:11:05.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:11:05.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:11:05.595 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:11:05.595 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:11:05.596 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:11:05.597 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:11:05.599 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:11:05.599 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:11:05.599 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:11:05.600 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:05.601 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:11:05.601 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:11:05.601 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:05.602 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:11:05.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:11:05.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-26T03:11:05.616 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:11:05.617 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.617 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:11:05.617 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.79512 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/ec-error-rollforward 0 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-26T03:11:05.619 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-26T03:11:05.620 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:11:05.620 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:11:05.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:11:05.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:11:05.620 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:11:05.621 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:11:05.621 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:11:05.622 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:11:05.623 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:11:05.624 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:11:05.624 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:11:05.625 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:11:05.626 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:05.626 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:11:05.626 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:11:05.626 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:05.627 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:11:05.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-26T03:11:05.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-26T03:11:05.630 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:11:05.630 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.630 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79512 2026-02-26T03:11:05.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.79512 2026-02-26T03:11:05.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:11:05.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:11:05.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-26T03:11:05.632 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-26T03:11:05.632 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T03:11:05.682 INFO:tasks.workunit:Running workunit osd/osd-bench.sh... 2026-02-26T03:11:05.683 DEBUG:teuthology.orchestra.run.trial123:workunit test osd/osd-bench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh 2026-02-26T03:11:05.735 INFO:tasks.workunit.client.0.trial123.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-26T03:11:05.738 INFO:tasks.workunit.client.0.trial123.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-bench 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:22: run: local dir=td/osd-bench 2026-02-26T03:11:05.739 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:23: run: shift 2026-02-26T03:11:05.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: export CEPH_MON=127.0.0.1:7106 2026-02-26T03:11:05.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: CEPH_MON=127.0.0.1:7106 2026-02-26T03:11:05.740 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:26: run: export CEPH_ARGS 2026-02-26T03:11:05.740 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: uuidgen 2026-02-26T03:11:05.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: CEPH_ARGS+='--fsid=a325dcf6-6354-42a1-8693-fcb2a4f7cac9 --auth-supported=none ' 2026-02-26T03:11:05.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7106 ' 2026-02-26T03:11:05.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:29: run: CEPH_ARGS+='--debug-bluestore 20 ' 2026-02-26T03:11:05.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: set 2026-02-26T03:11:05.741 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: local funcs=TEST_bench 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:32: run: for func in $funcs 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:33: run: setup td/osd-bench 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bench 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bench 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:11:05.743 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:11:05.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:11:05.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:11:05.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:11:05.744 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:11:05.745 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:11:05.746 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:11:05.746 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:11:05.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:11:05.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:11:05.747 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:11:05.748 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:05.748 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:11:05.749 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:11:05.749 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:05.750 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:11:05.751 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:11:05.751 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-26T03:11:05.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:11:05.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.753 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:05.753 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.89955 2026-02-26T03:11:05.754 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:11:05.754 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:11:05.754 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bench 2026-02-26T03:11:05.756 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:11:05.756 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.756 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:05.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.89955 2026-02-26T03:11:05.758 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:11:05.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-26T03:11:05.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-26T03:11:05.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:11:05.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bench 1' TERM HUP INT 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:34: run: TEST_bench td/osd-bench 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:40: TEST_bench: local dir=td/osd-bench 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:42: TEST_bench: run_mon td/osd-bench a 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bench 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bench/a 2026-02-26T03:11:05.760 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bench/a --run-dir=td/osd-bench 2026-02-26T03:11:05.784 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:11:05.784 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:05.785 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:05.785 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:05.785 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.785 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:05.785 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.89955/$cluster-$name.asok' 2026-02-26T03:11:05.785 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bench/a '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.89955/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bench/log --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:11:05.815 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:11:05.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:11:05.818 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:11:05.819 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:11:05.819 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.819 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:05.819 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.89955/ceph-mon.a.asok 2026-02-26T03:11:05.819 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:11:05.819 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.89955/ceph-mon.a.asok config get fsid 2026-02-26T03:11:05.874 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:11:05.874 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:11:05.874 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:05.875 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.89955/ceph-mon.a.asok 2026-02-26T03:11:05.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:11:05.876 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.89955/ceph-mon.a.asok config get mon_host 2026-02-26T03:11:05.928 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:43: TEST_bench: run_mgr td/osd-bench x 2026-02-26T03:11:05.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bench 2026-02-26T03:11:05.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:11:05.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:11:05.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:11:05.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bench/x 2026-02-26T03:11:05.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:11:06.035 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:11:06.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:06.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:06.036 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:06.036 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:06.036 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:06.036 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.89955/$cluster-$name.asok' 2026-02-26T03:11:06.037 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:11:06.037 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bench/x '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.89955/$cluster-$name.asok' --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:11:06.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:44: TEST_bench: run_osd td/osd-bench 0 2026-02-26T03:11:06.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bench 2026-02-26T03:11:06.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:11:06.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:11:06.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:11:06.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bench/0 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a325dcf6-6354-42a1-8693-fcb2a4f7cac9 --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 ' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bench/0' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bench/0/journal' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bench' 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:06.058 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.89955/$cluster-$name.asok' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.89955/$cluster-$name.asok' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bench/$name.log' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bench/$name.pid' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:11:06.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:11:06.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:11:06.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:11:06.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:11:06.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:11:06.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bench/0 2026-02-26T03:11:06.060 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:11:06.061 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=311b546d-3b2a-4cbb-9d4a-053bb1d35583 2026-02-26T03:11:06.061 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 311b546d-3b2a-4cbb-9d4a-053bb1d35583' 2026-02-26T03:11:06.061 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 311b546d-3b2a-4cbb-9d4a-053bb1d35583 2026-02-26T03:11:06.061 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:11:06.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDKuZ9ppc0YBBAA57ANJipsPSW5/Uzao/NH4w== 2026-02-26T03:11:06.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDKuZ9ppc0YBBAA57ANJipsPSW5/Uzao/NH4w=="}' 2026-02-26T03:11:06.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 311b546d-3b2a-4cbb-9d4a-053bb1d35583 -i td/osd-bench/0/new.json 2026-02-26T03:11:06.179 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:11:06.187 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bench/0/new.json 2026-02-26T03:11:06.188 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a325dcf6-6354-42a1-8693-fcb2a4f7cac9 --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.89955/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKuZ9ppc0YBBAA57ANJipsPSW5/Uzao/NH4w== --osd-uuid 311b546d-3b2a-4cbb-9d4a-053bb1d35583 2026-02-26T03:11:06.203 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:06.200+0000 7fb833c3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:06.205 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:06.200+0000 7fb833c3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:06.206 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:06.204+0000 7fb833c3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:06.207 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:06.204+0000 7fb833c3e8c0 -1 bdev(0x557ed4197c00 td/osd-bench/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:11:06.207 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:06.204+0000 7fb833c3e8c0 -1 bluestore(td/osd-bench/0) _read_fsid unparsable uuid 2026-02-26T03:11:08.854 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bench/0/keyring 2026-02-26T03:11:08.855 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:11:08.855 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:11:08.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:11:08.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bench/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:11:09.118 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:11:09.119 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:11:09.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a325dcf6-6354-42a1-8693-fcb2a4f7cac9 --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.89955/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:11:09.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:11:09.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:11:09.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:11:09.139 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:09.136+0000 7f5710d418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:09.141 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:09.136+0000 7f5710d418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:09.142 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:09.140+0000 7f5710d418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:09.319 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:11:09.320 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:11:09.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:09.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:09.527 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:10.528 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:10.529 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:10.529 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:11:10.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:11:10.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:10.530 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:10.726 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:10.864 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:10.860+0000 7f5710d418c0 -1 Falling back to public interface 2026-02-26T03:11:11.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:11.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:11.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:11:11.728 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:11:11.729 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:11.729 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:11.920 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:12.922 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:12.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:12.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:11:12.923 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:11:12.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:12.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:13.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:13.874 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:13.872+0000 7f5710d418c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:11:14.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:14.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:14.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:11:14.121 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:11:14.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:14.122 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:14.337 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:15.338 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:15.338 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:15.338 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:11:15.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:11:15.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:15.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:15.536 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/850434721,v1:127.0.0.1:6803/850434721] [v2:127.0.0.1:6804/850434721,v1:127.0.0.1:6805/850434721] exists,up 311b546d-3b2a-4cbb-9d4a-053bb1d35583 2026-02-26T03:11:15.537 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:11:15.537 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:11:15.537 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:11:15.537 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: CEPH_ARGS= 2026-02-26T03:11:15.537 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: ceph-conf --show-config-value osd_bench_small_size_max_iops 2026-02-26T03:11:15.545 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: local osd_bench_small_size_max_iops=100 2026-02-26T03:11:15.546 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: CEPH_ARGS= 2026-02-26T03:11:15.546 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: ceph-conf --show-config-value osd_bench_large_size_max_throughput 2026-02-26T03:11:15.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: local osd_bench_large_size_max_throughput=104857600 2026-02-26T03:11:15.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: CEPH_ARGS= 2026-02-26T03:11:15.559 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: ceph-conf --show-config-value osd_bench_max_block_size 2026-02-26T03:11:15.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: local osd_bench_max_block_size=67108864 2026-02-26T03:11:15.571 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: CEPH_ARGS= 2026-02-26T03:11:15.571 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: ceph-conf --show-config-value osd_bench_duration 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: local osd_bench_duration=30 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:58: TEST_bench: expect_failure td/osd-bench osd_bench_max_block_size ceph tell osd.0 bench 1024 67108865 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=osd_bench_max_block_size 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-26T03:11:15.585 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 1024 67108865 2026-02-26T03:11:15.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-26T03:11:15.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-26T03:11:15.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet osd_bench_max_block_size td/osd-bench/out 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:64: TEST_bench: local bsize=1024 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:65: TEST_bench: local max_count=3072000 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:66: TEST_bench: expect_failure td/osd-bench bench_small_size_max_iops ceph tell osd.0 bench 3072001 1024 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=bench_small_size_max_iops 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-26T03:11:15.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 3072001 1024 2026-02-26T03:11:15.726 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-26T03:11:15.726 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-26T03:11:15.726 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet bench_small_size_max_iops td/osd-bench/out 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:72: TEST_bench: local bsize=1048577 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:73: TEST_bench: local max_count=3145728000 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:74: TEST_bench: expect_failure td/osd-bench osd_bench_large_size_max_throughput ceph tell osd.0 bench 3145728001 1048577 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=osd_bench_large_size_max_throughput 2026-02-26T03:11:15.727 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-26T03:11:15.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-26T03:11:15.728 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 3145728001 1048577 2026-02-26T03:11:15.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-26T03:11:15.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-26T03:11:15.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet osd_bench_large_size_max_throughput td/osd-bench/out 2026-02-26T03:11:15.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-26T03:11:15.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:80: TEST_bench: ceph tell osd.0 bench 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout: "bytes_written": 1073741824, 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout: "blocksize": 4194304, 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout: "elapsed_sec": 0.73100723899999998, 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout: "bytes_per_sec": 1468852518.4358673, 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout: "iops": 350.20173035523112 2026-02-26T03:11:17.484 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:11:17.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:84: TEST_bench: ceph tell osd.0 bench 10 14456 4444 3 2026-02-26T03:11:17.567 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:11:17.567 INFO:tasks.workunit.client.0.trial123.stdout: "bytes_written": 10, 2026-02-26T03:11:17.567 INFO:tasks.workunit.client.0.trial123.stdout: "blocksize": 14456, 2026-02-26T03:11:17.568 INFO:tasks.workunit.client.0.trial123.stdout: "elapsed_sec": 0.00039016499999999999, 2026-02-26T03:11:17.568 INFO:tasks.workunit.client.0.trial123.stdout: "bytes_per_sec": 25630.18210244384, 2026-02-26T03:11:17.568 INFO:tasks.workunit.client.0.trial123.stdout: "iops": 1.7729788394053569 2026-02-26T03:11:17.568 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:11:17.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:90: TEST_bench: ceph tell osd.0 bench 1 14456 2026-02-26T03:11:17.648 INFO:tasks.workunit.client.0.trial123.stdout:{ 2026-02-26T03:11:17.649 INFO:tasks.workunit.client.0.trial123.stdout: "bytes_written": 1, 2026-02-26T03:11:17.649 INFO:tasks.workunit.client.0.trial123.stdout: "blocksize": 14456, 2026-02-26T03:11:17.649 INFO:tasks.workunit.client.0.trial123.stdout: "elapsed_sec": 0.00052353299999999996, 2026-02-26T03:11:17.649 INFO:tasks.workunit.client.0.trial123.stdout: "bytes_per_sec": 1910.0992678589507, 2026-02-26T03:11:17.649 INFO:tasks.workunit.client.0.trial123.stdout: "iops": 0.13213193607214657 2026-02-26T03:11:17.649 INFO:tasks.workunit.client.0.trial123.stdout:} 2026-02-26T03:11:17.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:35: run: teardown td/osd-bench 2026-02-26T03:11:17.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-26T03:11:17.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:11:17.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-26T03:11:17.657 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:11:17.657 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:11:17.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:11:17.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:11:17.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:11:17.764 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:11:17.764 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:11:17.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:11:17.765 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:11:17.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:11:17.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:11:17.766 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:11:17.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:17.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:11:17.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:11:17.768 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:17.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:11:17.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:11:17.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-26T03:11:17.804 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:11:17.805 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:17.805 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:17.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.89955 2026-02-26T03:11:17.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:11:17.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:11:17.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-26T03:11:17.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-bench 0 2026-02-26T03:11:17.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-26T03:11:17.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-26T03:11:17.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-26T03:11:17.808 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:11:17.808 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:11:17.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:11:17.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:11:17.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:11:17.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:11:17.810 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:11:17.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:11:17.810 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:11:17.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:11:17.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:11:17.812 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:11:17.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:17.813 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:11:17.814 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:11:17.814 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:17.814 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:11:17.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-26T03:11:17.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-26T03:11:17.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:11:17.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:17.817 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.89955 2026-02-26T03:11:17.817 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.89955 2026-02-26T03:11:17.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:11:17.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:11:17.819 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-26T03:11:17.820 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-26T03:11:17.820 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T03:11:17.869 INFO:tasks.workunit:Running workunit osd/osd-bluefs-volume-ops.sh... 2026-02-26T03:11:17.869 DEBUG:teuthology.orchestra.run.trial123:workunit test osd/osd-bluefs-volume-ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh 2026-02-26T03:11:17.923 INFO:tasks.workunit.client.0.trial123.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-26T03:11:17.929 INFO:tasks.workunit.client.0.trial123.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-26T03:11:17.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T03:11:17.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-26T03:11:17.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-26T03:11:17.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-bluefs-volume-ops 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:8: run: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:9: run: shift 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: set 2026-02-26T03:11:17.930 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: local 'funcs=TEST_bluestore 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:TEST_bluestore2 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:TEST_bluestore_expand' 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:11:17.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-26T03:11:17.933 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:11:17.933 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:11:17.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:11:17.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:11:17.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:11:17.933 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:11:17.934 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:11:17.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:11:17.935 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:11:17.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:11:17.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:11:17.937 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:11:17.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:17.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:11:17.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:11:17.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:11:17.939 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:11:17.940 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:11:17.940 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-26T03:11:17.941 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:11:17.942 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:17.942 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:17.942 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92727 2026-02-26T03:11:17.943 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:11:17.943 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:11:17.943 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-26T03:11:17.945 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:11:17.945 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:17.945 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:17.945 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92727 2026-02-26T03:11:17.946 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:11:17.947 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-26T03:11:17.947 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-26T03:11:17.947 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:11:17.948 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-26T03:11:17.948 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore td/osd-bluefs-volume-ops 2026-02-26T03:11:17.948 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:20: TEST_bluestore: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:17.948 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: ulimit -n 2026-02-26T03:11:17.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: local flimit=4096 2026-02-26T03:11:17.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:23: TEST_bluestore: '[' 4096 -lt 1536 ']' 2026-02-26T03:11:17.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: export CEPH_MON=127.0.0.1:7146 2026-02-26T03:11:17.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: CEPH_MON=127.0.0.1:7146 2026-02-26T03:11:17.949 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:27: TEST_bluestore: export CEPH_ARGS 2026-02-26T03:11:17.949 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: uuidgen 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: CEPH_ARGS+='--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none ' 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:29: TEST_bluestore: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:30: TEST_bluestore: CEPH_ARGS+='--bluestore_block_size=2147483648 ' 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:31: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:32: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:33: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_size=536870912 ' 2026-02-26T03:11:17.951 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:34: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_create=true ' 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:35: TEST_bluestore: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:37: TEST_bluestore: CEPH_ARGS+='--bluestore_allocation_from_file=0 ' 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:39: TEST_bluestore: run_mon td/osd-bluefs-volume-ops a 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-26T03:11:17.952 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:17.973 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:11:17.973 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:17.973 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:17.974 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:17.974 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:17.974 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:17.974 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:17.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:11:18.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:11:18.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:11:18.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:11:18.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:11:18.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:11:18.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:11:18.005 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:11:18.005 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:11:18.005 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:11:18.006 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:11:18.006 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:18.006 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:18.006 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92727/ceph-mon.a.asok 2026-02-26T03:11:18.006 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:11:18.006 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92727/ceph-mon.a.asok config get fsid 2026-02-26T03:11:18.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:11:18.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:11:18.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:11:18.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:11:18.062 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:11:18.063 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:11:18.063 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:11:18.063 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:11:18.064 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:11:18.064 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:18.064 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:18.064 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92727/ceph-mon.a.asok 2026-02-26T03:11:18.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:11:18.064 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92727/ceph-mon.a.asok config get mon_host 2026-02-26T03:11:18.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:40: TEST_bluestore: run_mgr td/osd-bluefs-volume-ops x 2026-02-26T03:11:18.116 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:18.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:11:18.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:11:18.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:11:18.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-26T03:11:18.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:11:18.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:11:18.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:18.230 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:18.230 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:18.230 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:18.230 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:18.231 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:18.231 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:11:18.232 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:41: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:11:18.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:18.249 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:11:18.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:11:18.251 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:11:18.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:11:18.252 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 de317b43-1037-4495-a180-7869bb77a958' 2026-02-26T03:11:18.252 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:11:18.252 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:11:18.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDWuZ9pBV68DxAA36kAnODhQ8BfUZsxH1E7aw== 2026-02-26T03:11:18.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDWuZ9pBV68DxAA36kAnODhQ8BfUZsxH1E7aw=="}' 2026-02-26T03:11:18.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new de317b43-1037-4495-a180-7869bb77a958 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-26T03:11:18.371 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:11:18.379 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-26T03:11:18.380 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWuZ9pBV68DxAA36kAnODhQ8BfUZsxH1E7aw== --osd-uuid de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:11:18.396 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:18.392+0000 7f4e3c1588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:18.397 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:18.392+0000 7f4e3c1588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:18.401 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:18.396+0000 7f4e3c1588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:18.401 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:18.396+0000 7f4e3c1588c0 -1 bdev(0x5564fbc7dc00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:11:18.401 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:18.396+0000 7f4e3c1588c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-26T03:11:25.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-26T03:11:25.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:11:25.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:11:25.020 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:11:25.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:11:25.287 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:11:25.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:11:25.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:11:25.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:11:25.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:11:25.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:11:25.310 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:25.308+0000 7f717cfa48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:25.311 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:25.308+0000 7f717cfa48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:25.313 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:25.308+0000 7f717cfa48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:25.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:11:25.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:11:25.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:11:25.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:11:25.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:11:25.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:25.487 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:11:25.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:11:25.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:25.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:25.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:26.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:26.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:26.679 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:11:26.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:11:26.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:26.680 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:26.872 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:27.300 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:27.295+0000 7f717cfa48c0 -1 Falling back to public interface 2026-02-26T03:11:27.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:27.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:27.874 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:11:27.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:11:27.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:27.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:28.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:29.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:29.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:29.068 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:11:29.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:11:29.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:29.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:29.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:30.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:30.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:30.264 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:11:30.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:11:30.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:30.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:30.454 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:31.455 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:31.455 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:31.456 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:11:31.456 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:11:31.456 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:31.457 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:31.652 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:32.601 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:32.599+0000 7f717cfa48c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:11:32.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:32.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:32.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:11:32.653 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:11:32.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:32.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:32.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:33.847 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:33.847 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:33.847 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:11:33.848 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:11:33.848 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:33.848 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:11:34.041 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3311547370,v1:127.0.0.1:6803/3311547370] [v2:127.0.0.1:6804/3311547370,v1:127.0.0.1:6805/3311547370] exists,up de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:11:34.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:11:34.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:11:34.042 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:11:34.042 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: osd_pid0=94205 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:43: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 1 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:11:34.044 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:11:34.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:34.045 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:34.045 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:34.045 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:34.045 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-26T03:11:34.046 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:11:34.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:11:34.047 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 c680af24-a2de-49d3-a24a-3a23e752bc3b' 2026-02-26T03:11:34.047 INFO:tasks.workunit.client.0.trial123.stdout:add osd1 c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:11:34.047 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:11:34.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDmuZ9pS7SPAxAA0gWCsIzSZ9W1Z7GNGxBk/A== 2026-02-26T03:11:34.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDmuZ9pS7SPAxAA0gWCsIzSZ9W1Z7GNGxBk/A=="}' 2026-02-26T03:11:34.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c680af24-a2de-49d3-a24a-3a23e752bc3b -i td/osd-bluefs-volume-ops/1/new.json 2026-02-26T03:11:34.255 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:11:34.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/1/new.json 2026-02-26T03:11:34.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDmuZ9pS7SPAxAA0gWCsIzSZ9W1Z7GNGxBk/A== --osd-uuid c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:11:34.290 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:34.287+0000 7f2c160178c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:34.291 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:34.291+0000 7f2c160178c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:34.292 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:34.291+0000 7f2c160178c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:34.293 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:34.291+0000 7f2c160178c0 -1 bdev(0x55673361fc00 td/osd-bluefs-volume-ops/1/block) open stat got: (1) Operation not permitted 2026-02-26T03:11:34.293 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:34.291+0000 7f2c160178c0 -1 bluestore(td/osd-bluefs-volume-ops/1) _read_fsid unparsable uuid 2026-02-26T03:11:40.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/1/keyring 2026-02-26T03:11:40.896 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:11:40.896 INFO:tasks.workunit.client.0.trial123.stdout:adding osd1 key to auth repository 2026-02-26T03:11:40.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-26T03:11:40.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:11:41.166 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:11:41.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-26T03:11:41.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:11:41.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:11:41.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:11:41.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:11:41.181 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:41.179+0000 7f3efb4dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:41.182 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:41.183+0000 7f3efb4dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:41.183 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:41.183+0000 7f3efb4dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:11:41.359 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:41.360 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:41.557 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:42.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:42.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:42.559 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:11:42.559 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:11:42.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:42.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:42.763 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:43.232 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:43.231+0000 7f3efb4dd8c0 -1 Falling back to public interface 2026-02-26T03:11:43.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:43.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:43.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:11:43.766 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:11:43.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:43.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:43.959 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:44.960 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:44.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:44.961 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:11:44.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:11:44.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:44.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:45.164 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:46.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:46.166 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:46.166 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:11:46.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:11:46.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:46.167 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:46.367 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:47.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:47.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:47.369 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:11:47.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:11:47.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:47.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:47.567 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:48.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:48.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:48.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:11:48.568 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:11:48.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:48.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:48.576 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:48.575+0000 7f3efb4dd8c0 -1 osd.1 0 log_to_monitors true 2026-02-26T03:11:48.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:49.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:49.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:49.770 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:11:49.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:11:49.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:49.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:11:49.966 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/190933008,v1:127.0.0.1:6811/190933008] [v2:127.0.0.1:6812/190933008,v1:127.0.0.1:6813/190933008] exists,up c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:11:49.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:11:49.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:11:49.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:11:49.967 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: osd_pid1=96601 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:45: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 2 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:11:49.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:11:49.970 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:11:49.971 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:11:49.972 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:11:49.973 INFO:tasks.workunit.client.0.trial123.stdout:add osd2 9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:11:49.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 9b394c64-ffbf-4836-8bb4-a52940a096d1' 2026-02-26T03:11:49.974 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:11:49.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD1uZ9p5z2oOhAA8HjePKGPG8Ap/GpRHFc6zg== 2026-02-26T03:11:49.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD1uZ9p5z2oOhAA8HjePKGPG8Ap/GpRHFc6zg=="}' 2026-02-26T03:11:49.983 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9b394c64-ffbf-4836-8bb4-a52940a096d1 -i td/osd-bluefs-volume-ops/2/new.json 2026-02-26T03:11:50.184 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:11:50.193 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/2/new.json 2026-02-26T03:11:50.194 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD1uZ9p5z2oOhAA8HjePKGPG8Ap/GpRHFc6zg== --osd-uuid 9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:11:50.210 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:50.211+0000 7efda59888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:50.211 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:50.211+0000 7efda59888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:50.212 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:50.211+0000 7efda59888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:50.212 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:50.211+0000 7efda59888c0 -1 bdev(0x561a32737c00 td/osd-bluefs-volume-ops/2/block) open stat got: (1) Operation not permitted 2026-02-26T03:11:50.212 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:50.211+0000 7efda59888c0 -1 bluestore(td/osd-bluefs-volume-ops/2) _read_fsid unparsable uuid 2026-02-26T03:11:56.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/2/keyring 2026-02-26T03:11:56.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:11:56.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-26T03:11:56.799 INFO:tasks.workunit.client.0.trial123.stdout:adding osd2 key to auth repository 2026-02-26T03:11:56.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:11:57.067 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:11:57.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-26T03:11:57.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:11:57.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:11:57.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:11:57.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:11:57.089 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:57.091+0000 7fed3bb098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:57.091 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:57.091+0000 7fed3bb098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:57.092 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:57.091+0000 7fed3bb098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:11:57.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-26T03:11:57.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:11:57.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:11:57.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:11:57.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:11:57.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:57.270 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:11:57.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:11:57.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:57.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:11:57.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:58.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:58.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:58.468 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:11:58.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:11:58.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:58.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:11:58.670 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:11:59.339 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:11:59.339+0000 7fed3bb098c0 -1 Falling back to public interface 2026-02-26T03:11:59.672 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:11:59.672 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:11:59.672 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:11:59.673 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:11:59.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:11:59.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:11:59.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:00.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:00.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:00.869 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:12:00.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:12:00.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:00.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:12:01.067 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:02.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:02.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:02.068 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:12:02.068 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:12:02.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:02.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:12:02.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:03.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:03.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:03.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:12:03.271 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:12:03.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:03.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:12:03.468 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:04.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:04.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:04.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:12:04.471 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:12:04.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:04.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:12:04.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:04.682 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:04.683+0000 7fed3bb098c0 -1 osd.2 0 log_to_monitors true 2026-02-26T03:12:05.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:05.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:05.667 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:12:05.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:12:05.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:05.668 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:12:05.859 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1765169862,v1:127.0.0.1:6819/1765169862] [v2:127.0.0.1:6820/1765169862,v1:127.0.0.1:6821/1765169862] exists,up 9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:12:05.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:12:05.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:12:05.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:12:05.860 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: osd_pid2=98907 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:47: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 3 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:12:05.861 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:12:05.862 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:12:05.862 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:12:05.862 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:12:05.862 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:12:05.862 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:12:05.862 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:12:05.863 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-26T03:12:05.864 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:12:05.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:12:05.866 INFO:tasks.workunit.client.0.trial123.stdout:add osd3 3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:12:05.867 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 3fdaeba0-d4a8-468f-beba-15fc2990afae' 2026-02-26T03:12:05.867 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:12:05.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAFup9pSTZsNBAAJG69P3eAgpRsyrfF1cVZ3Q== 2026-02-26T03:12:05.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAFup9pSTZsNBAAJG69P3eAgpRsyrfF1cVZ3Q=="}' 2026-02-26T03:12:05.878 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3fdaeba0-d4a8-468f-beba-15fc2990afae -i td/osd-bluefs-volume-ops/3/new.json 2026-02-26T03:12:06.078 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:12:06.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/3/new.json 2026-02-26T03:12:06.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAFup9pSTZsNBAAJG69P3eAgpRsyrfF1cVZ3Q== --osd-uuid 3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:12:06.103 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:06.103+0000 7fa5dba958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:12:06.104 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:06.103+0000 7fa5dba958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:12:06.105 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:06.107+0000 7fa5dba958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:12:06.106 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:06.107+0000 7fa5dba958c0 -1 bdev(0x55e03dd75c00 td/osd-bluefs-volume-ops/3/block) open stat got: (1) Operation not permitted 2026-02-26T03:12:06.106 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:06.107+0000 7fa5dba958c0 -1 bluestore(td/osd-bluefs-volume-ops/3) _read_fsid unparsable uuid 2026-02-26T03:12:12.700 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/3/keyring 2026-02-26T03:12:12.701 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:12:12.701 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-26T03:12:12.702 INFO:tasks.workunit.client.0.trial123.stdout:adding osd3 key to auth repository 2026-02-26T03:12:12.702 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:12:12.973 INFO:tasks.workunit.client.0.trial123.stdout:start osd.3 2026-02-26T03:12:12.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-26T03:12:12.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:12:12.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:12:12.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:12:12.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:12:12.996 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:12.995+0000 7fe6c2db28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:12:12.998 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:12.999+0000 7fe6c2db28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:12:12.999 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:12.999+0000 7fe6c2db28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:13.169 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:12:13.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:12:13.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:13.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:13.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:14.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:14.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:14.371 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:12:14.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:12:14.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:14.372 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:14.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:15.036 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:15.035+0000 7fe6c2db28c0 -1 Falling back to public interface 2026-02-26T03:12:15.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:15.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:15.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:12:15.571 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:12:15.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:15.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:15.777 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:16.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:16.779 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:16.779 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:12:16.780 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:12:16.780 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:16.780 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:16.974 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:17.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:17.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:17.975 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:12:17.975 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:12:17.976 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:17.976 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:18.172 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:19.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:19.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:19.173 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:12:19.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:12:19.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:19.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:19.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:20.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:20.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:20.370 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:12:20.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:12:20.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:20.370 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:20.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:12:20.634 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:12:20.635+0000 7fe6c2db28c0 -1 osd.3 0 log_to_monitors true 2026-02-26T03:12:21.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:12:21.573 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:12:21.574 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:12:21.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:12:21.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:12:21.574 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:12:21.770 INFO:tasks.workunit.client.0.trial123.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2875544375,v1:127.0.0.1:6827/2875544375] [v2:127.0.0.1:6828/2875544375,v1:127.0.0.1:6829/2875544375] exists,up 3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:12:21.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:12:21.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:12:21.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:12:21.771 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-26T03:12:21.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: osd_pid3=101242 2026-02-26T03:12:21.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:50: TEST_bluestore: sleep 5 2026-02-26T03:12:26.773 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:52: TEST_bluestore: create_pool foo 16 2026-02-26T03:12:26.774 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-26T03:12:27.024 INFO:tasks.workunit.client.0.trial123.stderr:pool 'foo' already exists 2026-02-26T03:12:27.033 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T03:12:28.035 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:55: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-26T03:12:28.055 INFO:tasks.workunit.client.0.trial123.stdout:hints = 1 2026-02-26T03:12:28.055 INFO:tasks.workunit.client.0.trial123.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-26T03:12:28.055 INFO:tasks.workunit.client.0.trial123.stdout:Object prefix: benchmark_data_trial123_102325 2026-02-26T03:12:28.055 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:12:28.055 INFO:tasks.workunit.client.0.trial123.stdout: 0 0 0 0 0 0 - 0 2026-02-26T03:12:29.055 INFO:tasks.workunit.client.0.trial123.stdout: 1 16 9324 9308 36.3565 36.3594 0.00166187 0.00171559 2026-02-26T03:12:30.055 INFO:tasks.workunit.client.0.trial123.stdout: 2 16 18723 18707 36.5343 36.7148 0.00178181 0.00170835 2026-02-26T03:12:31.055 INFO:tasks.workunit.client.0.trial123.stdout: 3 16 28130 28114 36.6037 36.7461 0.00183304 0.00170548 2026-02-26T03:12:32.055 INFO:tasks.workunit.client.0.trial123.stdout: 4 16 37526 37510 36.6279 36.7031 0.00177997 0.00170458 2026-02-26T03:12:33.055 INFO:tasks.workunit.client.0.trial123.stdout: 5 16 46888 46872 36.6158 36.5703 0.00150444 0.00170519 2026-02-26T03:12:34.055 INFO:tasks.workunit.client.0.trial123.stdout: 6 16 56306 56290 36.6437 36.7891 0.00151404 0.0017039 2026-02-26T03:12:35.055 INFO:tasks.workunit.client.0.trial123.stdout: 7 16 65857 65841 36.7383 37.3086 0.00210853 0.00169962 2026-02-26T03:12:36.055 INFO:tasks.workunit.client.0.trial123.stdout: 8 16 75571 75555 36.8888 37.9453 0.00174417 0.00169273 2026-02-26T03:12:37.055 INFO:tasks.workunit.client.0.trial123.stdout: 9 16 86024 86008 37.3265 40.832 0.00144681 0.00167292 2026-02-26T03:12:38.055 INFO:tasks.workunit.client.0.trial123.stdout: 10 16 95783 95767 37.4057 38.1211 0.00110531 0.00166937 2026-02-26T03:12:39.055 INFO:tasks.workunit.client.0.trial123.stdout: 11 16 106120 106104 37.6757 40.3789 0.00191736 0.00165738 2026-02-26T03:12:40.056 INFO:tasks.workunit.client.0.trial123.stdout: 12 16 116327 116311 37.8584 39.8711 0.00149547 0.00164931 2026-02-26T03:12:41.056 INFO:tasks.workunit.client.0.trial123.stdout: 13 16 126177 126161 37.9057 38.4766 0.00133485 0.00164732 2026-02-26T03:12:42.056 INFO:tasks.workunit.client.0.trial123.stdout: 14 16 136194 136178 37.9929 39.1289 0.00145796 0.00164355 2026-02-26T03:12:43.056 INFO:tasks.workunit.client.0.trial123.stdout: 15 16 145470 145454 37.8755 36.2344 0.00159449 0.00164863 2026-02-26T03:12:44.056 INFO:tasks.workunit.client.0.trial123.stdout: 16 16 153927 153911 37.5728 33.0352 0.00146039 0.00166193 2026-02-26T03:12:45.056 INFO:tasks.workunit.client.0.trial123.stdout: 17 16 162665 162649 37.3703 34.1328 0.00175534 0.0016709 2026-02-26T03:12:46.056 INFO:tasks.workunit.client.0.trial123.stdout: 18 16 170763 170747 37.0514 31.6328 0.00296335 0.00168535 2026-02-26T03:12:47.056 INFO:tasks.workunit.client.0.trial123.stdout: 19 16 178862 178846 36.7663 31.6367 0.00238766 0.00169844 2026-02-26T03:12:48.056 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T03:12:48.058962+0000 min lat: 0.000639819 max lat: 0.0118439 avg lat: 0.00171152 2026-02-26T03:12:48.056 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:12:48.057 INFO:tasks.workunit.client.0.trial123.stdout: 20 16 186836 186820 36.4853 31.1484 0.0016381 0.00171152 2026-02-26T03:12:49.056 INFO:tasks.workunit.client.0.trial123.stdout: 21 16 195164 195148 36.2969 32.5312 0.00136943 0.00172044 2026-02-26T03:12:50.056 INFO:tasks.workunit.client.0.trial123.stdout: 22 16 203560 203544 36.1377 32.7969 0.00158518 0.00172804 2026-02-26T03:12:51.056 INFO:tasks.workunit.client.0.trial123.stdout: 23 16 211792 211776 35.9645 32.1562 0.00189664 0.00173634 2026-02-26T03:12:52.056 INFO:tasks.workunit.client.0.trial123.stdout: 24 16 220209 220193 35.8358 32.8789 0.00161541 0.00174258 2026-02-26T03:12:53.057 INFO:tasks.workunit.client.0.trial123.stdout: 25 16 228478 228462 35.6943 32.3008 0.00222714 0.0017495 2026-02-26T03:12:54.057 INFO:tasks.workunit.client.0.trial123.stdout: 26 16 236586 236570 35.5395 31.6719 0.00192245 0.00175712 2026-02-26T03:12:55.056 INFO:tasks.workunit.client.0.trial123.stdout: 27 16 245038 245022 35.4459 33.0156 0.00173368 0.00176179 2026-02-26T03:12:56.057 INFO:tasks.workunit.client.0.trial123.stdout: 28 16 253259 253243 35.3268 32.1133 0.00181471 0.00176773 2026-02-26T03:12:57.057 INFO:tasks.workunit.client.0.trial123.stdout: 29 16 258949 258933 34.875 22.2266 0.0519176 0.00178519 2026-02-26T03:12:58.057 INFO:tasks.workunit.client.0.trial123.stdout: 30 11 265285 265274 34.5381 24.7695 0.0657826 0.00180661 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Total time run: 30.0698 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Total writes made: 265285 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Write size: 4096 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Object size: 4096 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Bandwidth (MB/sec): 34.4622 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Bandwidth: 4.2151 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Max bandwidth (MB/sec): 40.832 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Min bandwidth (MB/sec): 22.2266 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Average IOPS: 8822 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Stddev IOPS: 1079.07 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Max IOPS: 10453 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Min IOPS: 5690 2026-02-26T03:12:59.057 INFO:tasks.workunit.client.0.trial123.stdout:Average Latency(s): 0.00181088 2026-02-26T03:12:59.058 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Latency(s): 0.00176481 2026-02-26T03:12:59.058 INFO:tasks.workunit.client.0.trial123.stdout:Max latency(s): 0.117587 2026-02-26T03:12:59.058 INFO:tasks.workunit.client.0.trial123.stdout:Min latency(s): 0.000639819 2026-02-26T03:12:59.063 INFO:tasks.workunit.client.0.trial123.stdout:after bench 2026-02-26T03:12:59.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:57: TEST_bluestore: echo 'after bench' 2026-02-26T03:12:59.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 94205 2026-02-26T03:12:59.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: sleep 1 2026-02-26T03:13:00.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 94205 2026-02-26T03:13:00.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 60: kill: (94205) - No such process 2026-02-26T03:13:00.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:61: TEST_bluestore: ceph osd down 0 2026-02-26T03:13:00.263 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:13:00.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 96601 2026-02-26T03:13:00.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: sleep 1 2026-02-26T03:13:01.274 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 96601 2026-02-26T03:13:01.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 62: kill: (96601) - No such process 2026-02-26T03:13:01.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:63: TEST_bluestore: ceph osd down 1 2026-02-26T03:13:01.471 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:13:01.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 98907 2026-02-26T03:13:01.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: sleep 1 2026-02-26T03:13:02.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 98907 2026-02-26T03:13:02.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 64: kill: (98907) - No such process 2026-02-26T03:13:02.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:65: TEST_bluestore: ceph osd down 2 2026-02-26T03:13:02.685 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:13:02.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 101242 2026-02-26T03:13:02.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: sleep 1 2026-02-26T03:13:03.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 101242 2026-02-26T03:13:03.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 66: kill: (101242) - No such process 2026-02-26T03:13:03.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:67: TEST_bluestore: ceph osd down 3 2026-02-26T03:13:04.276 INFO:tasks.workunit.client.0.trial123.stderr:osd.3 is already down. 2026-02-26T03:13:04.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:70: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:13:09.453 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:13:09.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:71: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:13:14.629 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:13:14.671 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:72: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:13:19.781 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:13:19.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:73: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:13:24.937 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:13:24.978 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:75: TEST_bluestore: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-26T03:13:24.980 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:76: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-26T03:13:24.997 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:0 : device size 0x20000000(512 MiB) : using 0x18500000(389 MiB) 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0x100000000(4 GiB) : using 0xb091a000(2.8 GiB) 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:Expanding DB/WAL... 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:0 : nothing to do, skipped 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:1 : nothing to do, skipped 2026-02-26T03:13:28.410 INFO:tasks.workunit.client.0.trial123.stdout:2 : Expanding to 0x100000000(4 GiB) 2026-02-26T03:13:28.411 INFO:tasks.workunit.client.0.trial123.stdout:2 : size updated to 0x100000000(4 GiB) 2026-02-26T03:13:33.398 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:77: TEST_bluestore: truncate td/osd-bluefs-volume-ops/1/block -s 11811160064 2026-02-26T03:13:33.399 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:78: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 bluefs-bdev-expand 2026-02-26T03:13:33.416 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:13:36.943 INFO:tasks.workunit.client.0.trial123.stdout:0 : device size 0x20000000(512 MiB) : using 0x18500000(389 MiB) 2026-02-26T03:13:36.943 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-26T03:13:36.943 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0x2c0000000(11 GiB) : using 0x2709c3000(9.8 GiB) 2026-02-26T03:13:36.943 INFO:tasks.workunit.client.0.trial123.stdout:Expanding DB/WAL... 2026-02-26T03:13:36.944 INFO:tasks.workunit.client.0.trial123.stdout:0 : nothing to do, skipped 2026-02-26T03:13:36.944 INFO:tasks.workunit.client.0.trial123.stdout:1 : nothing to do, skipped 2026-02-26T03:13:36.944 INFO:tasks.workunit.client.0.trial123.stdout:2 : Expanding to 0x2c0000000(11 GiB) 2026-02-26T03:13:36.944 INFO:tasks.workunit.client.0.trial123.stdout:2 : size updated to 0x2c0000000(11 GiB) 2026-02-26T03:13:41.575 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:79: TEST_bluestore: truncate td/osd-bluefs-volume-ops/2/block -s 4295099392 2026-02-26T03:13:41.576 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:80: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 bluefs-bdev-expand 2026-02-26T03:13:41.595 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:13:45.098 INFO:tasks.workunit.client.0.trial123.stdout:0 : device size 0x20000000(512 MiB) : using 0x18a00000(394 MiB) 2026-02-26T03:13:45.098 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-26T03:13:45.098 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0x100020000(4.0 GiB) : using 0xb09d1000(2.8 GiB) 2026-02-26T03:13:45.099 INFO:tasks.workunit.client.0.trial123.stdout:Expanding DB/WAL... 2026-02-26T03:13:45.099 INFO:tasks.workunit.client.0.trial123.stdout:0 : nothing to do, skipped 2026-02-26T03:13:45.099 INFO:tasks.workunit.client.0.trial123.stdout:1 : nothing to do, skipped 2026-02-26T03:13:45.099 INFO:tasks.workunit.client.0.trial123.stdout:2 : Expanding to 0x100020000(4.0 GiB) 2026-02-26T03:13:45.100 INFO:tasks.workunit.client.0.trial123.stdout:2 : size updated to 0x100020000(4.0 GiB) 2026-02-26T03:13:50.039 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:81: TEST_bluestore: truncate td/osd-bluefs-volume-ops/3/block -s 4293918720 2026-02-26T03:13:50.040 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:82: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 bluefs-bdev-expand 2026-02-26T03:13:50.054 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:13:53.241 INFO:tasks.workunit.client.0.trial123.stdout:0 : device size 0x20000000(512 MiB) : using 0x18a00000(394 MiB) 2026-02-26T03:13:53.241 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-26T03:13:53.241 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0xfff00000(4.0 GiB) : using 0xb0820000(2.8 GiB) 2026-02-26T03:13:53.241 INFO:tasks.workunit.client.0.trial123.stdout:Expanding DB/WAL... 2026-02-26T03:13:53.241 INFO:tasks.workunit.client.0.trial123.stdout:0 : nothing to do, skipped 2026-02-26T03:13:53.242 INFO:tasks.workunit.client.0.trial123.stdout:1 : nothing to do, skipped 2026-02-26T03:13:53.242 INFO:tasks.workunit.client.0.trial123.stdout:2 : Expanding to 0xfff00000(4.0 GiB) 2026-02-26T03:13:53.242 INFO:tasks.workunit.client.0.trial123.stdout:2 : size updated to 0xfff00000(4.0 GiB) 2026-02-26T03:13:57.973 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:85: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:14:00.841 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:00.883 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:86: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:14:03.741 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:03.783 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:87: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:14:06.640 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:06.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:88: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:14:09.545 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:09.586 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:90: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-26T03:14:09.605 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:10.703 INFO:tasks.workunit.client.0.trial123.stdout:0 : device size 0x20000000(512 MiB) : using 0xc00000(12 MiB) 2026-02-26T03:14:10.704 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0x3e00000(62 MiB) 2026-02-26T03:14:10.704 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0x100000000(4 GiB) : using 0x3091a000(777 MiB) 2026-02-26T03:14:11.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:92: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/block.db --command bluefs-bdev-migrate 2026-02-26T03:14:11.682 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:14.609 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-26T03:14:14.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:97: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:14:16.968 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:17.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:100: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-26T03:14:17.029 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:19.952 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-26T03:14:19.959 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:105: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:14:22.341 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:22.383 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:108: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block.wal --devs-source td/osd-bluefs-volume-ops/2/block.db --dev-target td/osd-bluefs-volume-ops/2/block --command bluefs-bdev-migrate 2026-02-26T03:14:22.402 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:25.073 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-26T03:14:25.073 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-26T03:14:25.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:114: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:14:26.936 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:26.979 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:117: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block.wal --command bluefs-bdev-migrate 2026-02-26T03:14:26.998 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:27.836 INFO:tasks.workunit.client.0.trial123.stderr:Migrate to WAL device isn't supported. 2026-02-26T03:14:27.839 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:123: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-26T03:14:27.839 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:126: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:14:30.697 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:30.736 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:129: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-26T03:14:30.755 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:33.664 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-26T03:14:33.669 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:134: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:14:36.045 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:36.086 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:136: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --dev-target td/osd-bluefs-volume-ops/3/block.db --command bluefs-bdev-migrate 2026-02-26T03:14:36.105 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:14:38.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:141: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:14:40.600 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:14:40.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:143: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:14:40.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:14:40.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:14:40.648 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:14:40.649 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:14:40.650 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:14:40.668 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:40.668+0000 7f42db9f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:40.670 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:40.668+0000 7f42db9f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:40.671 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:40.668+0000 7f42db9f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:40.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:14:40.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:14:40.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:14:40.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:14:40.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:14:40.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:40.844 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:14:40.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:14:40.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:40.845 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:41.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:42.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:42.045 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:42.045 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:14:42.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:14:42.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:42.046 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:42.202 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:42.200+0000 7f42db9f08c0 -1 Falling back to public interface 2026-02-26T03:14:42.248 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:43.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:43.249 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:43.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:14:43.250 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:14:43.250 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:43.251 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:43.449 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:44.450 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:44.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:44.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:14:44.451 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:14:44.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:44.452 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:44.653 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:45.654 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:45.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:45.655 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:14:45.655 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:14:45.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:45.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:45.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:46.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:46.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:46.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:14:46.858 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:14:46.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:46.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:47.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:47.424 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:47.420+0000 7f42db9f08c0 -1 osd.0 22 log_to_monitors true 2026-02-26T03:14:48.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:48.059 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:48.059 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:14:48.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:14:48.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:48.060 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:14:48.263 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 31 up_thru 31 down_at 23 last_clean_interval [5,22) [v2:127.0.0.1:6802/1295973216,v1:127.0.0.1:6803/1295973216] [v2:127.0.0.1:6804/1295973216,v1:127.0.0.1:6805/1295973216] exists,up de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:14:48.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:14:48.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:14:48.265 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:14:48.265 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:14:48.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: osd_pid0=112973 2026-02-26T03:14:48.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:145: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-26T03:14:48.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:14:48.270 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:14:48.271 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:14:48.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:14:48.273 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-26T03:14:48.274 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:14:48.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:14:48.275 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:14:48.276 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-26T03:14:48.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:14:48.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:14:48.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:14:48.276 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:14:48.285 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:48.284+0000 7fca8e3be8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:48.287 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:48.284+0000 7fca8e3be8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:48.288 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:48.284+0000 7fca8e3be8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:48.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:14:48.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:14:48.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:14:48.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:14:48.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:14:48.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:48.470 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:14:48.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:14:48.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:48.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:48.672 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:49.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:49.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:49.674 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:14:49.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:14:49.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:49.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:49.793 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:49.792+0000 7fca8e3be8c0 -1 Falling back to public interface 2026-02-26T03:14:49.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:50.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:50.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:50.870 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:14:50.870 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:14:50.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:50.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:51.062 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:52.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:52.064 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:52.065 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:14:52.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:14:52.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:52.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:52.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:53.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:53.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:53.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:14:53.263 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:14:53.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:53.264 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:53.461 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:54.462 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:54.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:54.463 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:14:54.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:14:54.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:54.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:54.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:55.077 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:55.076+0000 7fca8e3be8c0 -1 osd.1 24 log_to_monitors true 2026-02-26T03:14:55.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:55.666 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:55.666 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:14:55.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:14:55.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:55.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:14:55.864 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 35 up_thru 35 down_at 25 last_clean_interval [9,24) [v2:127.0.0.1:6810/1601073797,v1:127.0.0.1:6811/1601073797] [v2:127.0.0.1:6812/1601073797,v1:127.0.0.1:6813/1601073797] exists,up c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:14:55.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:14:55.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:14:55.864 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:14:55.864 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: osd_pid1=114013 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:147: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:14:55.866 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:14:55.867 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:14:55.867 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:14:55.867 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:14:55.867 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:14:55.868 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-26T03:14:55.869 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:14:55.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:14:55.869 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:14:55.870 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-26T03:14:55.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:14:55.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:14:55.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:14:55.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:14:55.888 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:55.888+0000 7f9a3882a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:55.889 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:55.888+0000 7f9a3882a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:55.890 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:55.888+0000 7f9a3882a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:14:56.069 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:14:56.070 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:14:56.070 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:14:56.070 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:14:56.070 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:14:56.070 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:56.070 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:14:56.071 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:14:56.071 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:56.071 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:14:56.271 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:57.162 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:14:57.160+0000 7f9a3882a8c0 -1 Falling back to public interface 2026-02-26T03:14:57.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:57.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:57.272 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:14:57.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:14:57.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:57.272 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:14:57.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:58.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:58.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:58.475 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:14:58.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:14:58.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:58.475 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:14:58.673 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:14:59.674 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:14:59.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:14:59.675 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:14:59.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:14:59.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:14:59.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:14:59.873 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:00.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:00.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:00.874 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:15:00.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:15:00.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:00.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:15:01.072 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:01.990 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:15:01.988+0000 7f9a3882a8c0 -1 osd.2 26 log_to_monitors true 2026-02-26T03:15:02.073 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:02.074 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:02.074 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:15:02.075 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:15:02.075 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:02.075 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:15:02.276 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 38 up_thru 38 down_at 27 last_clean_interval [14,26) [v2:127.0.0.1:6818/3296261812,v1:127.0.0.1:6819/3296261812] [v2:127.0.0.1:6820/3296261812,v1:127.0.0.1:6821/3296261812] exists,up 9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:15:02.277 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:15:02.277 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:15:02.277 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:15:02.277 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-26T03:15:02.280 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: osd_pid2=115053 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:149: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:15:02.281 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:15:02.282 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:15:02.283 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-26T03:15:02.284 INFO:tasks.workunit.client.0.trial123.stdout:start osd.3 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:15:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:15:02.305 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:15:02.304+0000 7f60eeeca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:15:02.306 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:15:02.304+0000 7f60eeeca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:15:02.307 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:15:02.304+0000 7f60eeeca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:15:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-26T03:15:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:15:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-26T03:15:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:15:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:15:02.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:02.484 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:15:02.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:15:02.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:02.485 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:02.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:03.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:03.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:03.678 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:15:03.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:15:03.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:03.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:03.826 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:15:03.824+0000 7f60eeeca8c0 -1 Falling back to public interface 2026-02-26T03:15:03.873 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:04.874 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:04.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:04.875 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:15:04.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:15:04.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:04.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:05.063 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:06.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:06.065 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:06.065 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:15:06.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:15:06.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:06.066 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:06.258 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:07.260 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:07.260 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:07.260 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:15:07.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:15:07.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:07.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:07.456 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:08.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:08.458 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:08.459 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:15:08.459 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:15:08.459 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:08.460 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:08.656 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:15:09.102 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:15:09.100+0000 7f60eeeca8c0 -1 osd.3 28 log_to_monitors true 2026-02-26T03:15:09.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:15:09.657 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:15:09.658 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:15:09.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:15:09.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:15:09.658 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:15:09.857 INFO:tasks.workunit.client.0.trial123.stdout:osd.3 up in weight 1 up_from 41 up_thru 41 down_at 29 last_clean_interval [19,28) [v2:127.0.0.1:6826/1835207500,v1:127.0.0.1:6827/1835207500] [v2:127.0.0.1:6828/1835207500,v1:127.0.0.1:6829/1835207500] exists,up 3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:15:09.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:15:09.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:15:09.858 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:15:09.858 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-26T03:15:09.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: osd_pid3=116066 2026-02-26T03:15:09.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:152: TEST_bluestore: wait_for_clean 2026-02-26T03:15:09.859 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:15:09.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:15:09.860 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:15:09.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:15:09.861 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:15:09.861 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:15:09.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:15:09.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:15:09.861 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:15:09.960 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:15:09.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:15:09.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:15:09.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:15:09.961 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:15:09.961 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:15:10.159 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:15:10.159 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:15:10.159 INFO:tasks.workunit.client.0.trial123.stderr:2 2026-02-26T03:15:10.159 INFO:tasks.workunit.client.0.trial123.stderr:3' 2026-02-26T03:15:10.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:15:10.160 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:15:10.160 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:15:10.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986183 2026-02-26T03:15:10.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986183 2026-02-26T03:15:10.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986183' 2026-02-26T03:15:10.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:15:10.236 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:15:10.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855366 2026-02-26T03:15:10.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855366 2026-02-26T03:15:10.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986183 1-150323855366' 2026-02-26T03:15:10.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:15:10.310 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:15:10.386 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757252 2026-02-26T03:15:10.386 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757252 2026-02-26T03:15:10.386 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986183 1-150323855366 2-163208757252' 2026-02-26T03:15:10.386 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:15:10.386 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-26T03:15:10.462 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659139 2026-02-26T03:15:10.462 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659139 2026-02-26T03:15:10.462 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986183 1-150323855366 2-163208757252 3-176093659139' 2026-02-26T03:15:10.463 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:15:10.463 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-133143986183 2026-02-26T03:15:10.463 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:15:10.464 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:15:10.465 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-133143986183 2026-02-26T03:15:10.465 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:15:10.466 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986183 2026-02-26T03:15:10.466 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 133143986183 2026-02-26T03:15:10.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 133143986183' 2026-02-26T03:15:10.467 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:15:10.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986183 -lt 133143986183 2026-02-26T03:15:10.679 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:15:10.680 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-150323855366 2026-02-26T03:15:10.680 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:15:10.681 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:15:10.682 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-150323855366 2026-02-26T03:15:10.682 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:15:10.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855366 2026-02-26T03:15:10.685 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 150323855366 2026-02-26T03:15:10.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 150323855366' 2026-02-26T03:15:10.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:15:10.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855366 -lt 150323855366 2026-02-26T03:15:10.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:15:10.880 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-163208757252 2026-02-26T03:15:10.880 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:15:10.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:15:10.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-163208757252 2026-02-26T03:15:10.883 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:15:10.884 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757252 2026-02-26T03:15:10.884 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 163208757252' 2026-02-26T03:15:10.884 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 163208757252 2026-02-26T03:15:10.885 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:15:11.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757252 -lt 163208757252 2026-02-26T03:15:11.083 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:15:11.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-176093659139 2026-02-26T03:15:11.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:15:11.085 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-26T03:15:11.086 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-176093659139 2026-02-26T03:15:11.086 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:15:11.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659139 2026-02-26T03:15:11.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 176093659139' 2026-02-26T03:15:11.088 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.3 seq 176093659139 2026-02-26T03:15:11.088 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-26T03:15:11.301 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-26T03:15:11.301 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:15:11.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:15:11.302 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:15:11.568 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:15:11.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-26T03:15:11.768 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:15:11.769 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:15:11.769 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:15:12.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-26T03:15:12.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:15:12.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:15:12.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:155: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-26T03:15:12.040 INFO:tasks.workunit.client.0.trial123.stdout:hints = 1 2026-02-26T03:15:12.040 INFO:tasks.workunit.client.0.trial123.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-26T03:15:12.040 INFO:tasks.workunit.client.0.trial123.stdout:Object prefix: benchmark_data_trial123_117450 2026-02-26T03:15:12.040 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:15:12.041 INFO:tasks.workunit.client.0.trial123.stdout: 0 0 0 0 0 0 - 0 2026-02-26T03:15:13.040 INFO:tasks.workunit.client.0.trial123.stdout: 1 16 8462 8446 32.9895 32.9922 0.00205593 0.00189157 2026-02-26T03:15:14.040 INFO:tasks.workunit.client.0.trial123.stdout: 2 16 16596 16580 32.3803 31.7734 0.00181908 0.00192784 2026-02-26T03:15:15.044 INFO:tasks.workunit.client.0.trial123.stdout: 3 16 23416 23400 30.4274 26.6406 0.0017172 0.00204918 2026-02-26T03:15:16.044 INFO:tasks.workunit.client.0.trial123.stdout: 4 16 31544 31528 30.7572 31.75 0.00139572 0.00203011 2026-02-26T03:15:17.044 INFO:tasks.workunit.client.0.trial123.stdout: 5 16 39824 39808 31.0738 32.3438 0.00171336 0.00200957 2026-02-26T03:15:18.044 INFO:tasks.workunit.client.0.trial123.stdout: 6 16 48038 48022 31.242 32.0859 0.00211712 0.0019988 2026-02-26T03:15:19.044 INFO:tasks.workunit.client.0.trial123.stdout: 7 16 56511 56495 31.5066 33.0977 0.00192431 0.00198211 2026-02-26T03:15:20.044 INFO:tasks.workunit.client.0.trial123.stdout: 8 16 64210 64194 31.3273 30.0742 0.0464793 0.00198968 2026-02-26T03:15:21.044 INFO:tasks.workunit.client.0.trial123.stdout: 9 16 65616 65600 28.4579 5.49219 0.00207417 0.00219453 2026-02-26T03:15:22.044 INFO:tasks.workunit.client.0.trial123.stdout: 10 16 73944 73928 28.8649 32.5312 0.00186365 0.00216369 2026-02-26T03:15:23.045 INFO:tasks.workunit.client.0.trial123.stdout: 11 16 82143 82127 29.152 32.0273 0.00208562 0.00214233 2026-02-26T03:15:24.045 INFO:tasks.workunit.client.0.trial123.stdout: 12 16 90327 90311 29.3865 31.9688 0.00129372 0.0021253 2026-02-26T03:15:25.045 INFO:tasks.workunit.client.0.trial123.stdout: 13 16 98559 98543 29.5993 32.1562 0.00192593 0.00211002 2026-02-26T03:15:26.045 INFO:tasks.workunit.client.0.trial123.stdout: 14 16 106716 106700 29.7608 31.8633 0.00181096 0.0020985 2026-02-26T03:15:27.045 INFO:tasks.workunit.client.0.trial123.stdout: 15 16 114973 114957 29.9268 32.2539 0.00163114 0.00208691 2026-02-26T03:15:28.045 INFO:tasks.workunit.client.0.trial123.stdout: 16 16 123273 123257 30.0825 32.4219 0.00173791 0.00207615 2026-02-26T03:15:29.045 INFO:tasks.workunit.client.0.trial123.stdout: 17 16 131399 131383 30.18 31.7422 0.00160962 0.00206932 2026-02-26T03:15:30.045 INFO:tasks.workunit.client.0.trial123.stdout: 18 16 139842 139826 30.3354 32.9805 0.0020219 0.00205887 2026-02-26T03:15:31.045 INFO:tasks.workunit.client.0.trial123.stdout: 19 16 147495 147479 30.3121 29.8945 0.0437291 0.00205401 2026-02-26T03:15:32.045 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T03:15:32.049270+0000 min lat: 0.000700302 max lat: 0.672254 avg lat: 0.0021615 2026-02-26T03:15:32.046 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:15:32.046 INFO:tasks.workunit.client.0.trial123.stdout: 20 16 147999 147983 28.8952 1.96875 0.00241524 0.0021615 2026-02-26T03:15:33.046 INFO:tasks.workunit.client.0.trial123.stdout: 21 16 156340 156324 29.0706 32.582 0.0019073 0.00214847 2026-02-26T03:15:34.045 INFO:tasks.workunit.client.0.trial123.stdout: 22 16 164553 164537 29.2074 32.082 0.00281948 0.0021384 2026-02-26T03:15:35.046 INFO:tasks.workunit.client.0.trial123.stdout: 23 16 172764 172748 29.3319 32.0742 0.00195214 0.00212933 2026-02-26T03:15:36.046 INFO:tasks.workunit.client.0.trial123.stdout: 24 16 181302 181286 29.4992 33.3516 0.00195722 0.00211724 2026-02-26T03:15:37.046 INFO:tasks.workunit.client.0.trial123.stdout: 25 16 189572 189556 29.6113 32.3047 0.00139069 0.00210926 2026-02-26T03:15:38.046 INFO:tasks.workunit.client.0.trial123.stdout: 26 16 190369 190353 28.5923 3.11328 0.137867 0.0021669 2026-02-26T03:15:39.046 INFO:tasks.workunit.client.0.trial123.stdout: 27 16 190460 190444 27.5467 0.355469 0.108477 0.0022597 2026-02-26T03:15:40.046 INFO:tasks.workunit.client.0.trial123.stdout: 28 16 198850 198834 27.7332 32.7734 0.0017771 0.00225219 2026-02-26T03:15:41.046 INFO:tasks.workunit.client.0.trial123.stdout: 29 16 207005 206989 27.8753 31.8555 0.0016302 0.00224069 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Total time run: 30.0014 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Total writes made: 215570 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Write size: 4096 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Object size: 4096 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Bandwidth (MB/sec): 28.0677 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Bandwidth: 10.3374 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Max bandwidth (MB/sec): 33.3516 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Min bandwidth (MB/sec): 0.355469 2026-02-26T03:15:42.047 INFO:tasks.workunit.client.0.trial123.stdout:Average IOPS: 7185 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Stddev IOPS: 2646.38 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Max IOPS: 8538 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Min IOPS: 91 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Average Latency(s): 0.00222535 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Latency(s): 0.00772501 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Max latency(s): 0.672254 2026-02-26T03:15:42.048 INFO:tasks.workunit.client.0.trial123.stdout:Min latency(s): 0.000700302 2026-02-26T03:15:42.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 112973 2026-02-26T03:15:42.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: sleep 1 2026-02-26T03:15:43.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 112973 2026-02-26T03:15:43.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 158: kill: (112973) - No such process 2026-02-26T03:15:43.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:159: TEST_bluestore: ceph osd down 0 2026-02-26T03:15:43.257 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:15:43.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 114013 2026-02-26T03:15:43.268 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: sleep 1 2026-02-26T03:15:44.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 114013 2026-02-26T03:15:44.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 160: kill: (114013) - No such process 2026-02-26T03:15:44.269 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:161: TEST_bluestore: ceph osd down 1 2026-02-26T03:15:44.470 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:15:44.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 115053 2026-02-26T03:15:44.479 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: sleep 1 2026-02-26T03:15:45.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 115053 2026-02-26T03:15:45.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 162: kill: (115053) - No such process 2026-02-26T03:15:45.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:163: TEST_bluestore: ceph osd down 2 2026-02-26T03:15:45.684 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:15:45.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 116066 2026-02-26T03:15:45.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: sleep 1 2026-02-26T03:15:46.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 116066 2026-02-26T03:15:46.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 164: kill: (116066) - No such process 2026-02-26T03:15:46.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:165: TEST_bluestore: ceph osd down 3 2026-02-26T03:15:46.949 INFO:tasks.workunit.client.0.trial123.stderr:osd.3 is already down. 2026-02-26T03:15:46.958 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:168: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:15:51.588 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:15:51.643 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:170: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal count=512 bs=1M 2026-02-26T03:15:51.965 INFO:tasks.workunit.client.0.trial123.stderr:512+0 records in 2026-02-26T03:15:51.965 INFO:tasks.workunit.client.0.trial123.stderr:512+0 records out 2026-02-26T03:15:51.965 INFO:tasks.workunit.client.0.trial123.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.320613 s, 1.7 GB/s 2026-02-26T03:15:51.966 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:171: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --dev-target td/osd-bluefs-volume-ops/0/wal --command bluefs-bdev-new-wal 2026-02-26T03:15:51.982 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:15:57.616 INFO:tasks.workunit.client.0.trial123.stdout:WAL device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal 2026-02-26T03:15:57.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:175: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:16:02.552 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:16:02.605 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:178: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:16:07.244 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:16:07.299 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:180: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db count=1024 bs=1M 2026-02-26T03:16:07.937 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records in 2026-02-26T03:16:07.938 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records out 2026-02-26T03:16:07.938 INFO:tasks.workunit.client.0.trial123.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.63724 s, 1.7 GB/s 2026-02-26T03:16:07.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:181: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --dev-target td/osd-bluefs-volume-ops/1/db --command bluefs-bdev-new-db 2026-02-26T03:16:07.955 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:13.096 INFO:tasks.workunit.client.0.trial123.stdout:DB device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db 2026-02-26T03:16:13.118 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:185: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block --dev-target td/osd-bluefs-volume-ops/1/block.db --command bluefs-bdev-migrate 2026-02-26T03:16:13.134 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:17.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:190: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:16:22.692 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:16:22.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:193: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:16:26.628 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:16:26.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:195: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-db 2026-02-26T03:16:26.695 INFO:tasks.workunit.client.0.trial123.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-26T03:16:26.695 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:30.289 INFO:tasks.workunit.client.0.trial123.stdout:DB device added 2026-02-26T03:16:30.313 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:198: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-wal 2026-02-26T03:16:30.330 INFO:tasks.workunit.client.0.trial123.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-26T03:16:30.330 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:34.212 INFO:tasks.workunit.client.0.trial123.stdout:WAL device added 2026-02-26T03:16:34.237 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:201: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/block.db --command bluefs-bdev-migrate 2026-02-26T03:16:34.253 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:39.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:206: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:16:43.821 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:16:43.871 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:209: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:16:48.489 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:16:48.539 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:211: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --command bluefs-bdev-new-wal 2026-02-26T03:16:48.558 INFO:tasks.workunit.client.0.trial123.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-26T03:16:48.558 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:52.684 INFO:tasks.workunit.client.0.trial123.stdout:WAL device added 2026-02-26T03:16:52.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:214: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-26T03:16:52.728 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:16:57.636 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block.db 2026-02-26T03:16:57.667 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:219: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:17:02.024 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:17:02.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:221: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:17:02.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:17:02.079 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:17:02.080 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:02.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:02.081 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:17:02.082 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:17:02.083 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:17:02.084 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:17:02.103 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:02.101+0000 7f9e2abff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:02.105 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:02.105+0000 7f9e2abff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:02.106 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:02.105+0000 7f9e2abff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:02.284 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:02.285 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:02.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:03.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:03.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:03.484 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:17:03.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:17:03.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:03.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:03.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:03.830 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:03.828+0000 7f9e2abff8c0 -1 Falling back to public interface 2026-02-26T03:17:04.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:04.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:04.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:17:04.686 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:17:04.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:04.686 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:04.889 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:05.891 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:05.891 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:05.891 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:17:05.891 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:17:05.891 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:05.891 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:06.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:07.100 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:07.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:07.101 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:17:07.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:17:07.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:07.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:07.293 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:08.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:08.295 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:08.296 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:17:08.296 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:17:08.297 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:08.297 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:08.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:09.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:09.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:09.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:17:09.503 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:17:09.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:09.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:09.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:10.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:10.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:10.694 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:17:10.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:17:10.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:10.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:10.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:11.898 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:11.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:11.899 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:17:11.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:17:11.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:11.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:12.091 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:13.092 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:13.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:13.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:17:13.093 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:17:13.093 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:13.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:13.289 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:14.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:14.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:14.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:17:14.290 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:17:14.290 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:14.291 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:14.412 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:14.408+0000 7f9e2abff8c0 -1 osd.0 43 log_to_monitors true 2026-02-26T03:17:14.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:15.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:15.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:15.486 INFO:tasks.workunit.client.0.trial123.stdout:11 2026-02-26T03:17:15.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-26T03:17:15.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:15.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:17:15.687 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 52 up_thru 52 down_at 44 last_clean_interval [31,43) [v2:127.0.0.1:6802/3848313300,v1:127.0.0.1:6803/3848313300] [v2:127.0.0.1:6804/3848313300,v1:127.0.0.1:6805/3848313300] exists,up de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:17:15.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:17:15.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:17:15.688 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:17:15.688 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:17:15.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: osd_pid0=123605 2026-02-26T03:17:15.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:223: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-26T03:17:15.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:17:15.691 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:17:15.692 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:17:15.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-26T03:17:15.694 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:17:15.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:17:15.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:17:15.696 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-26T03:17:15.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:17:15.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:17:15.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:17:15.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:17:15.710 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:15.708+0000 7f55b0b4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:15.712 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:15.708+0000 7f55b0b4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:15.713 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:15.712+0000 7f55b0b4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:15.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:17:15.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:17:15.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:17:15.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:17:15.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:17:15.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:15.893 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:17:15.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:17:15.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:15.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:16.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:17.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:17.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:17.089 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:17:17.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:17:17.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:17.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:17.286 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:17.467 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:17.464+0000 7f55b0b4c8c0 -1 Falling back to public interface 2026-02-26T03:17:18.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:18.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:18.287 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:17:18.287 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:17:18.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:18.288 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:18.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:19.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:19.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:19.488 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:17:19.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:17:19.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:19.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:19.682 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:20.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:20.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:20.684 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:17:20.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:17:20.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:20.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:20.890 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:21.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:21.892 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:21.892 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:17:21.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:17:21.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:21.893 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:22.095 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:23.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:23.097 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:23.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:17:23.098 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:17:23.098 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:23.099 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:23.298 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:24.299 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:24.299 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:24.299 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:17:24.300 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:17:24.300 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:24.300 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:24.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:25.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:25.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:25.494 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:17:25.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:17:25.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:25.495 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:25.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:26.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:26.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:26.696 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:17:26.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:17:26.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:26.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:26.897 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:27.737 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:27.736+0000 7f55b0b4c8c0 -1 osd.1 45 log_to_monitors true 2026-02-26T03:17:27.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:27.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:27.899 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:17:27.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:17:27.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:27.900 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:17:28.101 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 46 last_clean_interval [35,45) [v2:127.0.0.1:6810/2232132839,v1:127.0.0.1:6811/2232132839] [v2:127.0.0.1:6812/2232132839,v1:127.0.0.1:6813/2232132839] exists,up c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:17:28.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:17:28.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:17:28.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:17:28.103 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: osd_pid1=124775 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:225: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-26T03:17:28.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:17:28.107 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:17:28.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:17:28.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:17:28.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-26T03:17:28.109 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:17:28.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:17:28.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:17:28.110 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-26T03:17:28.110 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:17:28.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:17:28.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:17:28.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:17:28.124 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:28.120+0000 7fea8c3a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:28.126 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:28.124+0000 7fea8c3a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:28.127 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:28.124+0000 7fea8c3a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:28.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:17:28.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:17:28.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:17:28.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:17:28.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:17:28.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:28.305 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:17:28.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:17:28.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:28.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:28.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:29.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:29.504 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:29.505 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:17:29.505 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:17:29.505 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:29.505 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:29.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:30.103 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:30.100+0000 7fea8c3a28c0 -1 Falling back to public interface 2026-02-26T03:17:30.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:30.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:30.709 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:17:30.709 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:17:30.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:30.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:30.909 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:31.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:31.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:31.910 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:17:31.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:17:31.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:31.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:32.109 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:33.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:33.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:33.111 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:17:33.112 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:17:33.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:33.112 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:33.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:34.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:34.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:34.307 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:17:34.307 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:17:34.307 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:34.308 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:34.508 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:35.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:35.509 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:35.510 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:17:35.510 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:17:35.510 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:35.510 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:35.705 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:36.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:36.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:36.707 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:17:36.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:17:36.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:36.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:36.910 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:37.912 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:37.912 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:37.913 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:17:37.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:17:37.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:37.914 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:38.106 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:39.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:39.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:39.107 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:17:39.107 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:17:39.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:39.108 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:39.318 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:40.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:40.320 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:40.320 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:17:40.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:17:40.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:40.322 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:40.514 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:40.689 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:40.691+0000 7fea8c3a28c0 -1 osd.2 47 log_to_monitors true 2026-02-26T03:17:41.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:41.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:41.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-26T03:17:41.516 INFO:tasks.workunit.client.0.trial123.stdout:11 2026-02-26T03:17:41.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:41.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:17:41.718 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 58 up_thru 58 down_at 48 last_clean_interval [38,47) [v2:127.0.0.1:6818/3049780419,v1:127.0.0.1:6819/3049780419] [v2:127.0.0.1:6820/3049780419,v1:127.0.0.1:6821/3049780419] exists,up 9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:17:41.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:17:41.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:17:41.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:17:41.719 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: osd_pid2=125920 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:227: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-26T03:17:41.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:17:41.721 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:17:41.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-26T03:17:41.723 INFO:tasks.workunit.client.0.trial123.stdout:start osd.3 2026-02-26T03:17:41.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-26T03:17:41.724 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:17:41.724 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-26T03:17:41.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-26T03:17:41.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:17:41.726 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:17:41.726 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:17:41.742 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:41.743+0000 7f1bd08aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:41.743 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:41.743+0000 7f1bd08aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:41.744 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:41.743+0000 7f1bd08aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:41.929 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:17:41.930 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:17:41.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:41.930 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:42.123 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:43.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:43.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:43.125 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:17:43.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:17:43.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:43.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:43.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:43.470 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:43.471+0000 7f1bd08aa8c0 -1 Falling back to public interface 2026-02-26T03:17:44.322 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:44.322 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:44.322 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:17:44.323 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:17:44.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:44.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:44.521 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:45.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:45.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:45.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:17:45.522 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:17:45.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:45.522 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:45.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:46.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:46.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:46.723 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:17:46.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:17:46.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:46.724 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:46.920 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:47.921 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:47.922 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:47.922 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:17:47.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:17:47.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:47.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:48.122 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:49.123 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:49.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:49.124 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:17:49.125 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:17:49.125 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:49.126 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:49.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:50.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:50.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:50.325 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:17:50.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:17:50.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:50.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:50.533 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:51.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:51.535 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:51.535 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:17:51.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:17:51.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:51.536 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:51.735 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:52.736 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:52.736 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:52.737 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:17:52.737 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:17:52.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:52.738 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:52.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:17:53.004 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:17:53.003+0000 7f1bd08aa8c0 -1 osd.3 49 log_to_monitors true 2026-02-26T03:17:53.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:17:53.936 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:17:53.937 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:17:53.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:17:53.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:17:53.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:17:54.131 INFO:tasks.workunit.client.0.trial123.stdout:osd.3 up in weight 1 up_from 61 up_thru 61 down_at 50 last_clean_interval [41,49) [v2:127.0.0.1:6826/4291059605,v1:127.0.0.1:6827/4291059605] [v2:127.0.0.1:6828/4291059605,v1:127.0.0.1:6829/4291059605] exists,up 3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:17:54.132 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:17:54.132 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:17:54.132 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:17:54.132 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-26T03:17:54.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: osd_pid3=127090 2026-02-26T03:17:54.133 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:231: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-26T03:17:54.151 INFO:tasks.workunit.client.0.trial123.stdout:hints = 1 2026-02-26T03:17:54.151 INFO:tasks.workunit.client.0.trial123.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-26T03:17:54.151 INFO:tasks.workunit.client.0.trial123.stdout:Object prefix: benchmark_data_trial123_128215 2026-02-26T03:17:54.151 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:17:54.151 INFO:tasks.workunit.client.0.trial123.stdout: 0 0 0 0 0 0 - 0 2026-02-26T03:17:55.151 INFO:tasks.workunit.client.0.trial123.stdout: 1 16 9600 9584 37.4348 37.4375 0.00170599 0.00166692 2026-02-26T03:17:56.151 INFO:tasks.workunit.client.0.trial123.stdout: 2 15 18767 18752 36.6223 35.8125 0.00133534 0.00170445 2026-02-26T03:17:57.151 INFO:tasks.workunit.client.0.trial123.stdout: 3 16 28163 28147 36.6471 36.6992 0.00173134 0.00170357 2026-02-26T03:17:58.151 INFO:tasks.workunit.client.0.trial123.stdout: 4 16 37196 37180 36.3059 35.2852 0.00166371 0.00171962 2026-02-26T03:17:59.151 INFO:tasks.workunit.client.0.trial123.stdout: 5 16 46693 46677 36.4637 37.0977 0.00184225 0.00171234 2026-02-26T03:18:00.151 INFO:tasks.workunit.client.0.trial123.stdout: 6 16 56158 56142 36.5481 36.9727 0.0011875 0.00170849 2026-02-26T03:18:01.151 INFO:tasks.workunit.client.0.trial123.stdout: 7 15 61802 61787 34.4768 22.0508 0.00134918 0.00181119 2026-02-26T03:18:02.151 INFO:tasks.workunit.client.0.trial123.stdout: 8 16 71556 71540 34.9291 38.0977 0.00093642 0.00178761 2026-02-26T03:18:03.151 INFO:tasks.workunit.client.0.trial123.stdout: 9 16 81223 81207 35.2435 37.7617 0.00155765 0.00177184 2026-02-26T03:18:04.152 INFO:tasks.workunit.client.0.trial123.stdout: 10 16 91111 91095 35.5813 38.625 0.0016765 0.00175499 2026-02-26T03:18:05.152 INFO:tasks.workunit.client.0.trial123.stdout: 11 16 100392 100376 35.6422 36.2539 0.00175872 0.00175199 2026-02-26T03:18:06.152 INFO:tasks.workunit.client.0.trial123.stdout: 12 16 109028 109012 35.483 33.7344 0.0174831 0.00175547 2026-02-26T03:18:07.152 INFO:tasks.workunit.client.0.trial123.stdout: 13 16 113217 113201 34.0122 16.3633 0.00129905 0.00183607 2026-02-26T03:18:08.152 INFO:tasks.workunit.client.0.trial123.stdout: 14 16 122380 122364 34.1392 35.793 0.00142212 0.00182923 2026-02-26T03:18:09.152 INFO:tasks.workunit.client.0.trial123.stdout: 15 16 131585 131569 34.2602 35.957 0.00174612 0.00182273 2026-02-26T03:18:10.152 INFO:tasks.workunit.client.0.trial123.stdout: 16 16 140874 140858 34.3866 36.2852 0.00155783 0.0018158 2026-02-26T03:18:11.152 INFO:tasks.workunit.client.0.trial123.stdout: 17 16 150061 150045 34.4747 35.8867 0.00124111 0.00181142 2026-02-26T03:18:12.152 INFO:tasks.workunit.client.0.trial123.stdout: 18 16 155608 155592 33.7631 21.668 0.130761 0.00183059 2026-02-26T03:18:13.152 INFO:tasks.workunit.client.0.trial123.stdout: 19 16 155662 155646 31.9972 0.210938 0.189741 0.00194081 2026-02-26T03:18:14.152 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T03:18:14.155265+0000 min lat: 0.000562308 max lat: 0.704593 avg lat: 0.00196929 2026-02-26T03:18:14.153 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:18:14.153 INFO:tasks.workunit.client.0.trial123.stdout: 20 16 162401 162385 31.7135 26.3242 0.00120081 0.00196929 2026-02-26T03:18:15.152 INFO:tasks.workunit.client.0.trial123.stdout: 21 15 171275 171260 31.854 34.668 0.00133428 0.00196058 2026-02-26T03:18:16.152 INFO:tasks.workunit.client.0.trial123.stdout: 22 16 180340 180324 32.0154 35.4062 0.00149499 0.00195071 2026-02-26T03:18:17.153 INFO:tasks.workunit.client.0.trial123.stdout: 23 16 189074 189058 32.1066 34.1172 0.00179815 0.00194517 2026-02-26T03:18:18.152 INFO:tasks.workunit.client.0.trial123.stdout: 24 16 198123 198107 32.2416 35.3477 0.00211179 0.00193699 2026-02-26T03:18:19.153 INFO:tasks.workunit.client.0.trial123.stdout: 25 16 207228 207212 32.3745 35.5664 0.00165779 0.00192904 2026-02-26T03:18:20.153 INFO:tasks.workunit.client.0.trial123.stdout: 26 16 216056 216040 32.4555 34.4844 0.00152088 0.00192426 2026-02-26T03:18:21.153 INFO:tasks.workunit.client.0.trial123.stdout: 27 16 224845 224829 32.5249 34.332 0.00206393 0.00192014 2026-02-26T03:18:22.153 INFO:tasks.workunit.client.0.trial123.stdout: 28 16 233819 233803 32.6152 35.0547 0.00152896 0.00191482 2026-02-26T03:18:23.153 INFO:tasks.workunit.client.0.trial123.stdout: 29 16 243039 243023 32.7323 36.0156 0.00141973 0.00190799 2026-02-26T03:18:24.153 INFO:tasks.workunit.client.0.trial123.stdout: 30 16 246678 246662 32.115 14.2148 0.0631861 0.00194217 2026-02-26T03:18:25.153 INFO:tasks.workunit.client.0.trial123.stdout:Total time run: 30.6859 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Total writes made: 246678 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Write size: 4096 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Object size: 4096 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Bandwidth (MB/sec): 31.4016 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Bandwidth: 8.74874 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Max bandwidth (MB/sec): 38.625 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Min bandwidth (MB/sec): 0.210938 2026-02-26T03:18:25.154 INFO:tasks.workunit.client.0.trial123.stdout:Average IOPS: 8038 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Stddev IOPS: 2239.68 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Max IOPS: 9888 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Min IOPS: 54 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Average Latency(s): 0.00197237 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Latency(s): 0.00811559 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Max latency(s): 0.70583 2026-02-26T03:18:25.155 INFO:tasks.workunit.client.0.trial123.stdout:Min latency(s): 0.000562308 2026-02-26T03:18:25.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 123605 2026-02-26T03:18:25.311 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: sleep 1 2026-02-26T03:18:26.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 123605 2026-02-26T03:18:26.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 234: kill: (123605) - No such process 2026-02-26T03:18:26.312 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:235: TEST_bluestore: ceph osd down 0 2026-02-26T03:18:26.683 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:18:26.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 124775 2026-02-26T03:18:26.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-26T03:18:27.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 124775 2026-02-26T03:18:27.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 236: kill: (124775) - No such process 2026-02-26T03:18:27.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:237: TEST_bluestore: ceph osd down 1 2026-02-26T03:18:27.895 INFO:tasks.workunit.client.0.trial123.stderr:osd.1 is already down. 2026-02-26T03:18:27.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 125920 2026-02-26T03:18:27.904 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: sleep 1 2026-02-26T03:18:28.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 125920 2026-02-26T03:18:28.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 238: kill: (125920) - No such process 2026-02-26T03:18:28.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:239: TEST_bluestore: ceph osd down 2 2026-02-26T03:18:29.109 INFO:tasks.workunit.client.0.trial123.stderr:osd.2 is already down. 2026-02-26T03:18:29.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 127090 2026-02-26T03:18:29.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: sleep 1 2026-02-26T03:18:30.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 127090 2026-02-26T03:18:30.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 240: kill: (127090) - No such process 2026-02-26T03:18:30.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:241: TEST_bluestore: ceph osd down 3 2026-02-26T03:18:30.371 INFO:tasks.workunit.client.0.trial123.stderr:osd.3 is already down. 2026-02-26T03:18:30.380 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:244: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:18:35.777 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:18:35.844 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:246: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/db2 count=1024 bs=1M 2026-02-26T03:18:36.482 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records in 2026-02-26T03:18:36.483 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records out 2026-02-26T03:18:36.483 INFO:tasks.workunit.client.0.trial123.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.63664 s, 1.7 GB/s 2026-02-26T03:18:36.483 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:247: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/db2 --command bluefs-bdev-migrate 2026-02-26T03:18:36.500 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:18:42.421 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-26T03:18:42.421 INFO:tasks.workunit.client.0.trial123.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/db2 2026-02-26T03:18:42.451 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:252: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:18:47.809 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:18:47.875 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:256: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal2 count=512 bs=1M 2026-02-26T03:18:48.213 INFO:tasks.workunit.client.0.trial123.stderr:512+0 records in 2026-02-26T03:18:48.214 INFO:tasks.workunit.client.0.trial123.stderr:512+0 records out 2026-02-26T03:18:48.214 INFO:tasks.workunit.client.0.trial123.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.337056 s, 1.6 GB/s 2026-02-26T03:18:48.214 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:257: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/wal2 --command bluefs-bdev-migrate 2026-02-26T03:18:48.230 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:18:53.905 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-26T03:18:53.905 INFO:tasks.workunit.client.0.trial123.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal2 2026-02-26T03:18:53.938 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:261: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/0/wal 2026-02-26T03:18:53.986 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:263: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:18:59.349 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:18:59.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:266: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:19:05.041 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:19:05.105 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:268: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db2 count=1024 bs=1M 2026-02-26T03:19:05.725 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records in 2026-02-26T03:19:05.725 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records out 2026-02-26T03:19:05.725 INFO:tasks.workunit.client.0.trial123.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.618577 s, 1.7 GB/s 2026-02-26T03:19:05.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:269: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --devs-source td/osd-bluefs-volume-ops/1/block.wal --dev-target td/osd-bluefs-volume-ops/1/db2 --command bluefs-bdev-migrate 2026-02-26T03:19:05.743 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:19:11.901 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-26T03:19:11.901 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/1/block.wal 2026-02-26T03:19:11.901 INFO:tasks.workunit.client.0.trial123.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db2 2026-02-26T03:19:11.934 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:275: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db 2026-02-26T03:19:12.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:277: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:19:16.925 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:19:16.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:279: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-26T03:19:17.012 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:19:21.389 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-26T03:19:21.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:284: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db2 2026-02-26T03:19:21.554 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:286: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-26T03:19:25.917 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:19:25.980 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:289: TEST_bluestore: ceph-objectstore-tool --type bluestore --data-path td/osd-bluefs-volume-ops/2 --op fsck --no-mon-config 2026-02-26T03:19:31.561 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:19:31.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:292: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-26T03:19:32.262 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records in 2026-02-26T03:19:32.263 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records out 2026-02-26T03:19:32.263 INFO:tasks.workunit.client.0.trial123.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.632297 s, 1.7 GB/s 2026-02-26T03:19:32.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:293: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-26T03:19:32.280 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:19:33.141 INFO:tasks.workunit.client.0.trial123.stderr:Unable to migrate Slow volume to new location, please allocate new DB or WAL with --bluefs-bdev-new-db(wal) command 2026-02-26T03:19:33.143 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:299: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-26T03:19:33.144 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:302: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:19:38.497 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:19:38.560 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:305: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-26T03:19:39.934 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records in 2026-02-26T03:19:39.935 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records out 2026-02-26T03:19:39.935 INFO:tasks.workunit.client.0.trial123.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.21738 s, 882 MB/s 2026-02-26T03:19:39.935 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:307: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --devs-source td/osd-bluefs-volume-ops/2/block.db --devs-source td/osd-bluefs-volume-ops/2/block.wal --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-26T03:19:39.947 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:19:45.353 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-26T03:19:45.354 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-26T03:19:45.354 INFO:tasks.workunit.client.0.trial123.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/2/db2 2026-02-26T03:19:45.387 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:314: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-26T03:19:50.505 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:19:50.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:317: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/3/wal2 count=1024 bs=1M 2026-02-26T03:19:51.201 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records in 2026-02-26T03:19:51.202 INFO:tasks.workunit.client.0.trial123.stderr:1024+0 records out 2026-02-26T03:19:51.202 INFO:tasks.workunit.client.0.trial123.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.630794 s, 1.7 GB/s 2026-02-26T03:19:51.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:319: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/wal2 --command bluefs-bdev-migrate 2026-02-26T03:19:51.214 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:19:56.841 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block 2026-02-26T03:19:56.841 INFO:tasks.workunit.client.0.trial123.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-26T03:19:56.841 INFO:tasks.workunit.client.0.trial123.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/3/wal2 2026-02-26T03:19:56.876 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:325: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-26T03:20:01.729 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:327: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:20:01.793 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:20:01.794 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:20:01.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:20:01.796 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:20:01.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:20:01.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:20:01.797 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-26T03:20:01.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:20:01.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:20:01.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:20:01.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:20:01.814 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:01.812+0000 7f7ecf5a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:01.816 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:01.812+0000 7f7ecf5a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:01.817 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:01.812+0000 7f7ecf5a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:02.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:02.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:03.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:03.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:03.202 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:20:03.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:20:03.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:03.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:03.402 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:03.543 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:03.540+0000 7f7ecf5a78c0 -1 Falling back to public interface 2026-02-26T03:20:04.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:04.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:04.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:20:04.404 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:20:04.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:04.405 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:04.601 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:05.602 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:05.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:05.603 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:20:05.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:20:05.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:05.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:05.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:06.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:06.806 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:20:06.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:06.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:20:06.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:06.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:07.008 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:08.009 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:08.010 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:08.010 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:20:08.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:20:08.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:08.011 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:08.210 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:09.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:09.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:09.212 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:20:09.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:20:09.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:09.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:09.414 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:10.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:10.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:10.416 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:20:10.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:20:10.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:10.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:10.625 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:11.626 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:11.627 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:11.627 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:20:11.627 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:20:11.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:11.628 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:11.828 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:12.829 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:12.830 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:20:12.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:12.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:20:12.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:12.830 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:13.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:14.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:14.029 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:20:14.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:14.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:20:14.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:14.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:14.228 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:15.017 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:15.019+0000 7f7ecf5a78c0 -1 osd.0 63 log_to_monitors true 2026-02-26T03:20:15.229 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:15.230 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:15.230 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-26T03:20:15.230 INFO:tasks.workunit.client.0.trial123.stdout:11 2026-02-26T03:20:15.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:15.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:20:15.426 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 73 up_thru 73 down_at 64 last_clean_interval [52,63) [v2:127.0.0.1:6802/45351877,v1:127.0.0.1:6803/45351877] [v2:127.0.0.1:6804/45351877,v1:127.0.0.1:6805/45351877] exists,up de317b43-1037-4495-a180-7869bb77a958 2026-02-26T03:20:15.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:20:15.427 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:20:15.427 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:20:15.427 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: osd_pid0=134410 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:329: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:20:15.429 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:20:15.430 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:20:15.430 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:20:15.430 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:20:15.430 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stdout:start osd.1 2026-02-26T03:20:15.431 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-26T03:20:15.432 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:20:15.432 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-26T03:20:15.433 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-26T03:20:15.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:20:15.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:20:15.434 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:20:15.450 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:15.451+0000 7f5f98c8b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:15.451 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:15.451+0000 7f5f98c8b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:15.452 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:15.451+0000 7f5f98c8b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:20:15.632 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:15.633 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:15.825 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:16.723 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:16.723+0000 7f5f98c8b8c0 -1 Falling back to public interface 2026-02-26T03:20:16.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:16.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:16.826 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:20:16.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:20:16.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:16.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:17.021 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:18.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:18.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:18.023 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:20:18.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:20:18.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:18.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:18.215 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:19.216 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:19.216 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:19.216 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:20:19.217 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:20:19.217 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:19.217 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:19.413 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:20.415 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:20.415 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:20:20.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:20.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:20:20.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:20.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:20.608 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:21.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:21.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:21.609 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:20:21.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:20:21.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:21.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:21.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:22.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:22.806 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:22.806 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:20:22.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:20:22.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:22.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:23.001 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:24.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:24.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:24.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:20:24.002 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:20:24.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:24.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:24.196 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:25.197 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:25.197 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:20:25.197 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:25.197 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:20:25.197 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:25.198 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:25.391 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:26.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:26.393 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:26.394 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:20:26.394 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:20:26.394 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:26.395 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:26.603 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:27.307 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:27.307+0000 7f5f98c8b8c0 -1 osd.1 66 log_to_monitors true 2026-02-26T03:20:27.604 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:27.605 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:20:27.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:27.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:20:27.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:27.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-26T03:20:27.801 INFO:tasks.workunit.client.0.trial123.stdout:osd.1 up in weight 1 up_from 76 up_thru 76 down_at 67 last_clean_interval [55,66) [v2:127.0.0.1:6810/511681974,v1:127.0.0.1:6811/511681974] [v2:127.0.0.1:6812/511681974,v1:127.0.0.1:6813/511681974] exists,up c680af24-a2de-49d3-a24a-3a23e752bc3b 2026-02-26T03:20:27.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:20:27.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:20:27.802 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:20:27.802 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-26T03:20:27.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: osd_pid1=135580 2026-02-26T03:20:27.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:331: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-26T03:20:27.804 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:20:27.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:20:27.806 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:20:27.806 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:20:27.806 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:20:27.806 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:20:27.806 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-26T03:20:27.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-26T03:20:27.808 INFO:tasks.workunit.client.0.trial123.stdout:start osd.2 2026-02-26T03:20:27.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:20:27.808 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-26T03:20:27.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-26T03:20:27.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:20:27.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:20:27.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:20:27.825 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:27.827+0000 7ff7235e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:27.826 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:27.827+0000 7ff7235e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:27.827 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:27.827+0000 7ff7235e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:28.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-26T03:20:28.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:20:28.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-26T03:20:28.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:20:28.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:20:28.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:28.003 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:20:28.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:20:28.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:28.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:28.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:29.206 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:29.206 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:29.206 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:20:29.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:20:29.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:29.207 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:29.315 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:29.315+0000 7ff7235e78c0 -1 Falling back to public interface 2026-02-26T03:20:29.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:30.406 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:30.406 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:30.406 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:20:30.407 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:20:30.407 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:30.407 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:30.605 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:31.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:31.607 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:31.607 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:20:31.608 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:20:31.608 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:31.608 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:31.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:32.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:32.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:32.810 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:20:32.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:20:32.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:32.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:33.005 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:34.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:34.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:34.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:20:34.006 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:20:34.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:34.006 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:34.201 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:35.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:35.202 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:35.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:20:35.203 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:20:35.203 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:35.204 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:35.401 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:36.402 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:36.403 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:36.403 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:20:36.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:20:36.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:36.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:36.605 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:37.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:37.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:37.606 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:20:37.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:20:37.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:37.606 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:37.798 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:38.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:38.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:38.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:20:38.800 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:20:38.801 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:38.801 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:38.994 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:39.995 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:39.995 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:39.995 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:20:39.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:20:39.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:39.996 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:40.211 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:40.376 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:40.375+0000 7ff7235e78c0 -1 osd.2 68 log_to_monitors true 2026-02-26T03:20:41.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:41.212 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:41.213 INFO:tasks.workunit.client.0.trial123.stdout:11 2026-02-26T03:20:41.213 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-26T03:20:41.214 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:41.214 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-26T03:20:41.415 INFO:tasks.workunit.client.0.trial123.stdout:osd.2 up in weight 1 up_from 80 up_thru 80 down_at 69 last_clean_interval [58,68) [v2:127.0.0.1:6818/1379594740,v1:127.0.0.1:6819/1379594740] [v2:127.0.0.1:6820/1379594740,v1:127.0.0.1:6821/1379594740] exists,up 9b394c64-ffbf-4836-8bb4-a52940a096d1 2026-02-26T03:20:41.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:20:41.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:20:41.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:20:41.416 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-26T03:20:41.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: osd_pid2=136715 2026-02-26T03:20:41.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:333: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-26T03:20:41.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:20:41.420 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:20:41.422 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:20:41.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:20:41.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-26T03:20:41.423 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-26T03:20:41.423 INFO:tasks.workunit.client.0.trial123.stdout:start osd.3 2026-02-26T03:20:41.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:20:41.424 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-26T03:20:41.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-26T03:20:41.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:20:41.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:20:41.424 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:20:41.438 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:41.439+0000 7f0e572218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:41.439 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:41.439+0000 7f0e572218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:41.440 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:41.439+0000 7f0e572218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:41.630 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:20:41.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:20:41.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:41.631 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:41.824 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:42.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:42.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:42.826 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:20:42.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:20:42.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:42.826 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:43.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:43.163 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:43.163+0000 7f0e572218c0 -1 Falling back to public interface 2026-02-26T03:20:44.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:44.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:44.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:20:44.023 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:20:44.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:44.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:44.220 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:45.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:45.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:45.222 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:20:45.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:20:45.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:45.222 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:45.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:46.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:46.420 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:46.420 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:20:46.421 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:20:46.421 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:46.421 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:46.615 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:47.616 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:47.616 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:47.616 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:20:47.616 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:20:47.616 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:47.616 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:47.821 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:48.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:48.822 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:20:48.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:48.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:20:48.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:48.824 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:49.020 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:50.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:50.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:50.022 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:20:50.023 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:20:50.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:50.023 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:50.221 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:51.223 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:51.223 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:20:51.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:51.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:20:51.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:51.224 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:51.415 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:52.416 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:52.417 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:20:52.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:52.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:20:52.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:52.418 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:52.611 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:53.612 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:53.613 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:20:53.613 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:53.614 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:20:53.614 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:53.614 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:53.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:20:54.225 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:20:54.222+0000 7f0e572218c0 -1 osd.3 70 log_to_monitors true 2026-02-26T03:20:54.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:20:54.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:20:54.808 INFO:tasks.workunit.client.0.trial123.stdout:11 2026-02-26T03:20:54.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-26T03:20:54.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:20:54.809 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-26T03:20:55.003 INFO:tasks.workunit.client.0.trial123.stdout:osd.3 up in weight 1 up_from 83 up_thru 83 down_at 71 last_clean_interval [61,70) [v2:127.0.0.1:6826/3791012838,v1:127.0.0.1:6827/3791012838] [v2:127.0.0.1:6828/3791012838,v1:127.0.0.1:6829/3791012838] exists,up 3fdaeba0-d4a8-468f-beba-15fc2990afae 2026-02-26T03:20:55.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:20:55.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:20:55.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:20:55.004 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-26T03:20:55.005 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: osd_pid3=137880 2026-02-26T03:20:55.005 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:337: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-26T03:20:55.024 INFO:tasks.workunit.client.0.trial123.stdout:hints = 1 2026-02-26T03:20:55.024 INFO:tasks.workunit.client.0.trial123.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-26T03:20:55.024 INFO:tasks.workunit.client.0.trial123.stdout:Object prefix: benchmark_data_trial123_139030 2026-02-26T03:20:55.024 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:20:55.024 INFO:tasks.workunit.client.0.trial123.stdout: 0 0 0 0 0 0 - 0 2026-02-26T03:20:56.024 INFO:tasks.workunit.client.0.trial123.stdout: 1 16 10071 10055 39.2741 39.2773 0.00163437 0.00158882 2026-02-26T03:20:57.024 INFO:tasks.workunit.client.0.trial123.stdout: 2 16 19989 19973 39.0067 38.7422 0.00140083 0.00159997 2026-02-26T03:20:58.024 INFO:tasks.workunit.client.0.trial123.stdout: 3 15 29947 29932 38.971 38.9023 0.00148123 0.00160167 2026-02-26T03:20:59.024 INFO:tasks.workunit.client.0.trial123.stdout: 4 16 37063 37047 36.1759 27.793 0.00157063 0.00172582 2026-02-26T03:21:00.024 INFO:tasks.workunit.client.0.trial123.stdout: 5 16 47171 47155 36.837 39.4844 0.00175873 0.00169486 2026-02-26T03:21:01.025 INFO:tasks.workunit.client.0.trial123.stdout: 6 16 57024 57008 37.1118 38.4883 0.00168098 0.0016823 2026-02-26T03:21:02.024 INFO:tasks.workunit.client.0.trial123.stdout: 7 16 67011 66995 37.3828 39.0117 0.00174253 0.00167019 2026-02-26T03:21:03.025 INFO:tasks.workunit.client.0.trial123.stdout: 8 15 77032 77017 37.6031 39.1484 0.000916076 0.00166051 2026-02-26T03:21:04.025 INFO:tasks.workunit.client.0.trial123.stdout: 9 16 87193 87177 37.8343 39.6875 0.00164879 0.00165035 2026-02-26T03:21:05.025 INFO:tasks.workunit.client.0.trial123.stdout: 10 16 92621 92605 36.1711 21.2031 0.0013087 0.00172628 2026-02-26T03:21:06.025 INFO:tasks.workunit.client.0.trial123.stdout: 11 16 102493 102477 36.3882 38.5625 0.00184907 0.00171602 2026-02-26T03:21:07.025 INFO:tasks.workunit.client.0.trial123.stdout: 12 16 112104 112088 36.4842 37.543 0.00171444 0.0017115 2026-02-26T03:21:08.025 INFO:tasks.workunit.client.0.trial123.stdout: 13 16 121371 121355 36.462 36.1992 0.00183729 0.00171253 2026-02-26T03:21:09.025 INFO:tasks.workunit.client.0.trial123.stdout: 14 16 130637 130621 36.4427 36.1953 0.00152908 0.00171347 2026-02-26T03:21:10.025 INFO:tasks.workunit.client.0.trial123.stdout: 15 16 140048 140032 36.4638 36.7617 0.00241037 0.00171248 2026-02-26T03:21:11.025 INFO:tasks.workunit.client.0.trial123.stdout: 16 16 149443 149427 36.4783 36.6992 0.00133905 0.00171178 2026-02-26T03:21:12.025 INFO:tasks.workunit.client.0.trial123.stdout: 17 16 158971 158955 36.5217 37.2188 0.00183195 0.00170979 2026-02-26T03:21:13.025 INFO:tasks.workunit.client.0.trial123.stdout: 18 16 168338 168322 36.5253 36.5898 0.0016824 0.00170957 2026-02-26T03:21:14.026 INFO:tasks.workunit.client.0.trial123.stdout: 19 16 177940 177924 36.5769 37.5078 0.00109762 0.00170716 2026-02-26T03:21:15.026 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T03:21:15.028515+0000 min lat: 0.000590613 max lat: 0.110773 avg lat: 0.00172587 2026-02-26T03:21:15.026 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:21:15.026 INFO:tasks.workunit.client.0.trial123.stdout: 20 16 185100 185084 36.1464 27.9688 0.0293202 0.00172587 2026-02-26T03:21:16.026 INFO:tasks.workunit.client.0.trial123.stdout: 21 16 185201 185185 34.4439 0.394531 0.0574874 0.00180172 2026-02-26T03:21:17.026 INFO:tasks.workunit.client.0.trial123.stdout: 22 16 189523 189507 33.6456 16.8828 0.0016416 0.00185606 2026-02-26T03:21:18.026 INFO:tasks.workunit.client.0.trial123.stdout: 23 16 198809 198793 33.7598 36.2734 0.00217664 0.00184977 2026-02-26T03:21:19.026 INFO:tasks.workunit.client.0.trial123.stdout: 24 16 208109 208093 33.8667 36.3281 0.0021389 0.00184394 2026-02-26T03:21:20.026 INFO:tasks.workunit.client.0.trial123.stdout: 25 16 217474 217458 33.9752 36.582 0.00120306 0.00183808 2026-02-26T03:21:21.026 INFO:tasks.workunit.client.0.trial123.stdout: 26 16 226825 226809 34.0732 36.5273 0.00146194 0.00183274 2026-02-26T03:21:22.026 INFO:tasks.workunit.client.0.trial123.stdout: 27 16 233107 233091 33.72 24.5391 0.148875 0.00184378 2026-02-26T03:21:23.037 INFO:tasks.workunit.client.0.trial123.stdout: 28 16 233201 233185 32.5289 0.367188 0.165495 0.00191065 2026-02-26T03:21:24.026 INFO:tasks.workunit.client.0.trial123.stdout: 29 16 233317 233301 31.4228 0.453125 0.120094 0.00198469 2026-02-26T03:21:25.026 INFO:tasks.workunit.client.0.trial123.stdout:Total time run: 30.0006 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Total writes made: 237120 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Write size: 4096 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Object size: 4096 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Bandwidth (MB/sec): 30.8744 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Bandwidth: 12.1513 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Max bandwidth (MB/sec): 39.6875 2026-02-26T03:21:25.027 INFO:tasks.workunit.client.0.trial123.stdout:Min bandwidth (MB/sec): 0.367188 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Average IOPS: 7903 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Stddev IOPS: 3110.74 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Max IOPS: 10160 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Min IOPS: 94 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Average Latency(s): 0.00202286 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Latency(s): 0.00761561 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Max latency(s): 0.303852 2026-02-26T03:21:25.028 INFO:tasks.workunit.client.0.trial123.stdout:Min latency(s): 0.000590613 2026-02-26T03:21:25.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:339: TEST_bluestore: wait_for_clean 2026-02-26T03:21:25.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:21:25.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:21:25.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:21:25.031 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:21:25.031 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:21:25.031 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:21:25.031 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:21:25.032 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:21:25.032 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:21:25.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:21:25.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:21:25.129 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:21:25.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:21:25.130 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:21:25.130 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:21:25.332 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-26T03:21:25.332 INFO:tasks.workunit.client.0.trial123.stderr:1 2026-02-26T03:21:25.332 INFO:tasks.workunit.client.0.trial123.stderr:2 2026-02-26T03:21:25.333 INFO:tasks.workunit.client.0.trial123.stderr:3' 2026-02-26T03:21:25.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:21:25.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:21:25.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:21:25.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612625 2026-02-26T03:21:25.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612625 2026-02-26T03:21:25.411 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-313532612625' 2026-02-26T03:21:25.412 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:21:25.412 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-26T03:21:25.490 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514510 2026-02-26T03:21:25.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514510 2026-02-26T03:21:25.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-313532612625 1-326417514510' 2026-02-26T03:21:25.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:21:25.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-26T03:21:25.564 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=343597383692 2026-02-26T03:21:25.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 343597383692 2026-02-26T03:21:25.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-313532612625 1-326417514510 2-343597383692' 2026-02-26T03:21:25.565 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:21:25.565 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-26T03:21:25.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285577 2026-02-26T03:21:25.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285577 2026-02-26T03:21:25.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-313532612625 1-326417514510 2-343597383692 3-356482285577' 2026-02-26T03:21:25.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:21:25.641 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-313532612625 2026-02-26T03:21:25.641 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:21:25.642 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:21:25.643 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-313532612625 2026-02-26T03:21:25.643 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:21:25.644 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612625 2026-02-26T03:21:25.645 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 313532612625 2026-02-26T03:21:25.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 313532612625' 2026-02-26T03:21:25.645 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:21:25.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612622 -lt 313532612625 2026-02-26T03:21:25.856 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:21:26.857 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:21:26.857 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:21:27.052 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612625 -lt 313532612625 2026-02-26T03:21:27.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:21:27.053 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-326417514510 2026-02-26T03:21:27.054 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:21:27.055 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-26T03:21:27.056 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-326417514510 2026-02-26T03:21:27.056 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:21:27.057 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514510 2026-02-26T03:21:27.058 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.1 seq 326417514510 2026-02-26T03:21:27.058 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 326417514510' 2026-02-26T03:21:27.058 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-26T03:21:27.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514510 -lt 326417514510 2026-02-26T03:21:27.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:21:27.262 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-343597383692 2026-02-26T03:21:27.262 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:21:27.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-26T03:21:27.264 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-343597383692 2026-02-26T03:21:27.264 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:21:27.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=343597383692 2026-02-26T03:21:27.266 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 343597383692' 2026-02-26T03:21:27.266 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.2 seq 343597383692 2026-02-26T03:21:27.266 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-26T03:21:27.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 343597383692 -lt 343597383692 2026-02-26T03:21:27.465 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:21:27.466 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-356482285577 2026-02-26T03:21:27.466 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:21:27.467 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-26T03:21:27.468 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-356482285577 2026-02-26T03:21:27.468 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:21:27.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285577 2026-02-26T03:21:27.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 356482285577' 2026-02-26T03:21:27.470 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.3 seq 356482285577 2026-02-26T03:21:27.470 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-26T03:21:27.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285577 -lt 356482285577 2026-02-26T03:21:27.675 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:21:27.676 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:21:27.676 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:21:27.943 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:21:27.944 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:21:28.163 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-26T03:21:28.163 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:21:28.164 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:21:28.164 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:21:28.429 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-26T03:21:28.430 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:21:28.430 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:21:28.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:21:28.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:21:28.430 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:21:28.741 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:21:28.742 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:21:28.743 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:21:28.744 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:21:28.745 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:21:28.745 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:21:28.746 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:21:28.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:21:28.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:21:28.747 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:21:28.748 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:21:28.749 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:21:28.750 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:21:28.750 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-26T03:21:34.474 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:21:34.474 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.474 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.474 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92727 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:21:34.476 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:21:34.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:21:34.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:21:34.477 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:21:34.480 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:21:34.480 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:21:34.481 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:21:34.481 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:21:34.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:21:34.482 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:21:34.482 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:21:34.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:21:34.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:21:34.484 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:21:34.484 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:21:34.486 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:21:34.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:21:34.487 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-26T03:21:34.488 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:21:34.488 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.488 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.488 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92727 2026-02-26T03:21:34.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:21:34.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:21:34.489 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-26T03:21:34.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:21:34.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.491 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.491 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92727 2026-02-26T03:21:34.493 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:21:34.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-26T03:21:34.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:21:34.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-26T03:21:34.493 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore2 td/osd-bluefs-volume-ops 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:343: TEST_bluestore2: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: ulimit -n 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: local flimit=4096 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:346: TEST_bluestore2: '[' 4096 -lt 1536 ']' 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: export CEPH_MON=127.0.0.1:7146 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: CEPH_MON=127.0.0.1:7146 2026-02-26T03:21:34.494 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:350: TEST_bluestore2: export CEPH_ARGS 2026-02-26T03:21:34.495 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: uuidgen 2026-02-26T03:21:34.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: CEPH_ARGS+='--fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none ' 2026-02-26T03:21:34.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:352: TEST_bluestore2: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-26T03:21:34.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:353: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-26T03:21:34.496 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:354: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:355: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:356: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:357: TEST_bluestore2: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:358: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:359: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:360: TEST_bluestore2: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:362: TEST_bluestore2: run_mon td/osd-bluefs-volume-ops a 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-26T03:21:34.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.519 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:21:34.519 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:21:34.519 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:21:34.520 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:21:34.520 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.520 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.520 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:21:34.520 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:21:34.548 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:21:34.549 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:21:34.549 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:21:34.549 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:21:34.549 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:21:34.549 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:21:34.550 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:21:34.550 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:21:34.550 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:21:34.550 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:21:34.550 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.550 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.551 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92727/ceph-mon.a.asok 2026-02-26T03:21:34.551 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:21:34.551 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92727/ceph-mon.a.asok config get fsid 2026-02-26T03:21:34.606 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:21:34.607 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:21:34.608 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:21:34.608 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.608 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.608 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92727/ceph-mon.a.asok 2026-02-26T03:21:34.608 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:21:34.608 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92727/ceph-mon.a.asok config get mon_host 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:363: TEST_bluestore2: run_mgr td/osd-bluefs-volume-ops x 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-26T03:21:34.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:21:34.773 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:21:34.773 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:21:34.773 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:21:34.774 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:21:34.774 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.774 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.774 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:21:34.775 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:21:34.776 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:21:34.795 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:364: TEST_bluestore2: run_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:21:34.796 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:21:34.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:21:34.799 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:21:34.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8e6214bf-0a48-43a0-a4b0-284b6c1155a0 2026-02-26T03:21:34.800 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 8e6214bf-0a48-43a0-a4b0-284b6c1155a0 2026-02-26T03:21:34.800 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 8e6214bf-0a48-43a0-a4b0-284b6c1155a0' 2026-02-26T03:21:34.800 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:21:34.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA+vJ9po1+EMBAAZr2HzHzA1t6UiPmymBvH9g== 2026-02-26T03:21:34.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA+vJ9po1+EMBAAZr2HzHzA1t6UiPmymBvH9g=="}' 2026-02-26T03:21:34.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8e6214bf-0a48-43a0-a4b0-284b6c1155a0 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-26T03:21:34.924 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:21:34.932 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-26T03:21:34.934 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+vJ9po1+EMBAAZr2HzHzA1t6UiPmymBvH9g== --osd-uuid 8e6214bf-0a48-43a0-a4b0-284b6c1155a0 2026-02-26T03:21:34.952 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:34.949+0000 7fe1df6748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:21:34.953 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:34.953+0000 7fe1df6748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:21:34.954 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:34.953+0000 7fe1df6748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:21:34.955 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:34.953+0000 7fe1df6748c0 -1 bdev(0x55f23e7bcc00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:21:34.955 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:34.953+0000 7fe1df6748c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-26T03:21:40.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-26T03:21:40.094 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:21:40.095 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:21:40.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:21:40.096 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:21:40.363 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:21:40.364 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:21:40.364 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:21:40.364 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:21:40.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:21:40.365 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:21:40.390 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:40.389+0000 7f7ce4e378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:21:40.391 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:40.389+0000 7f7ce4e378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:21:40.393 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:40.389+0000 7f7ce4e378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:21:40.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:21:40.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:21:40.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:21:40.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:21:40.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:21:40.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:40.570 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:21:40.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:21:40.570 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:40.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:40.765 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:21:41.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:21:41.766 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:41.767 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:21:41.767 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:21:41.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:41.768 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:41.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:21:42.363 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:42.361+0000 7f7ce4e378c0 -1 Falling back to public interface 2026-02-26T03:21:42.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:21:42.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:42.970 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:21:42.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:21:42.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:42.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:43.169 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:21:44.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:21:44.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:44.170 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:21:44.171 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:21:44.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:44.171 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:44.366 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:21:45.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:21:45.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:45.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:21:45.369 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:21:45.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:45.369 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:45.569 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:21:46.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:21:46.571 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:46.571 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:21:46.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:21:46.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:46.572 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:46.769 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:21:46.966 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:21:46.965+0000 7f7ce4e378c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:21:47.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:21:47.770 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:21:47.771 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:21:47.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:21:47.771 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:21:47.772 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:21:47.967 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4132461239,v1:127.0.0.1:6803/4132461239] [v2:127.0.0.1:6804/4132461239,v1:127.0.0.1:6805/4132461239] exists,up 8e6214bf-0a48-43a0-a4b0-284b6c1155a0 2026-02-26T03:21:47.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:21:47.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:21:47.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:21:47.969 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:21:47.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: osd_pid0=140966 2026-02-26T03:21:47.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:367: TEST_bluestore2: sleep 5 2026-02-26T03:21:52.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:368: TEST_bluestore2: create_pool foo 16 2026-02-26T03:21:52.971 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-26T03:21:53.223 INFO:tasks.workunit.client.0.trial123.stderr:pool 'foo' already exists 2026-02-26T03:21:53.234 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T03:21:54.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:370: TEST_bluestore2: retry = 0 2026-02-26T03:21:54.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 370: retry: command not found 2026-02-26T03:21:54.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:371: TEST_bluestore2: [[ '' -le 5 ]] 2026-02-26T03:21:54.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:373: TEST_bluestore2: timeout 60 rados bench -p foo 10 write --write-omap --no-cleanup 2026-02-26T03:21:54.257 INFO:tasks.workunit.client.0.trial123.stdout:hints = 1 2026-02-26T03:21:54.258 INFO:tasks.workunit.client.0.trial123.stdout:Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects 2026-02-26T03:21:54.258 INFO:tasks.workunit.client.0.trial123.stdout:Object prefix: benchmark_data_trial123_143723 2026-02-26T03:21:54.293 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:21:54.293 INFO:tasks.workunit.client.0.trial123.stdout: 0 0 0 0 0 0 - 0 2026-02-26T03:21:55.293 INFO:tasks.workunit.client.0.trial123.stdout: 1 16 179 163 651.94 652 0.0861028 0.0954175 2026-02-26T03:21:56.293 INFO:tasks.workunit.client.0.trial123.stdout: 2 16 371 355 709.938 768 0.0674068 0.0880987 2026-02-26T03:21:57.294 INFO:tasks.workunit.client.0.trial123.stdout: 3 16 579 563 750.604 832 0.0764725 0.0839974 2026-02-26T03:21:58.293 INFO:tasks.workunit.client.0.trial123.stdout: 4 16 778 762 761.936 796 0.0906079 0.0829743 2026-02-26T03:21:59.294 INFO:tasks.workunit.client.0.trial123.stdout: 5 16 1002 986 788.735 896 0.0828949 0.0805984 2026-02-26T03:22:00.294 INFO:tasks.workunit.client.0.trial123.stdout: 6 16 1203 1187 791.267 804 0.0823927 0.080525 2026-02-26T03:22:01.294 INFO:tasks.workunit.client.0.trial123.stdout: 7 16 1386 1370 782.79 732 0.0789843 0.0814759 2026-02-26T03:22:02.294 INFO:tasks.workunit.client.0.trial123.stdout: 8 16 1594 1578 788.931 832 0.0692208 0.080931 2026-02-26T03:22:03.294 INFO:tasks.workunit.client.0.trial123.stdout: 9 16 1802 1786 793.708 832 0.089621 0.0803911 2026-02-26T03:22:04.294 INFO:tasks.workunit.client.0.trial123.stdout: 10 16 1994 1978 791.128 768 0.0816253 0.0806688 2026-02-26T03:22:05.294 INFO:tasks.workunit.client.0.trial123.stdout:Total time run: 10.0742 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Total writes made: 1994 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Write size: 4194304 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Object size: 4194304 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Bandwidth (MB/sec): 791.726 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Bandwidth: 66.7679 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Max bandwidth (MB/sec): 896 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Min bandwidth (MB/sec): 652 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Average IOPS: 197 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Stddev IOPS: 16.692 2026-02-26T03:22:05.295 INFO:tasks.workunit.client.0.trial123.stdout:Max IOPS: 224 2026-02-26T03:22:05.296 INFO:tasks.workunit.client.0.trial123.stdout:Min IOPS: 163 2026-02-26T03:22:05.296 INFO:tasks.workunit.client.0.trial123.stdout:Average Latency(s): 0.080681 2026-02-26T03:22:05.296 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Latency(s): 0.0105695 2026-02-26T03:22:05.296 INFO:tasks.workunit.client.0.trial123.stdout:Max latency(s): 0.132646 2026-02-26T03:22:05.296 INFO:tasks.workunit.client.0.trial123.stdout:Min latency(s): 0.0345766 2026-02-26T03:22:05.307 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:376: TEST_bluestore2: sleep 10 2026-02-26T03:22:15.310 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: jq .bluefs.db_used_bytes 2026-02-26T03:22:15.310 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-26T03:22:15.392 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: db_used=865075200 2026-02-26T03:22:15.393 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-26T03:22:15.393 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: jq .bluefs.slow_used_bytes 2026-02-26T03:22:15.469 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: spilled_over=120848384 2026-02-26T03:22:15.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:380: TEST_bluestore2: (( retry+=1 )) 2026-02-26T03:22:15.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: test 120848384 -eq 0 2026-02-26T03:22:15.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: break 2026-02-26T03:22:15.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:383: TEST_bluestore2: test 120848384 -gt 0 2026-02-26T03:22:15.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 140966 2026-02-26T03:22:15.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: sleep 1 2026-02-26T03:22:16.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 140966 2026-02-26T03:22:16.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 385: kill: (140966) - No such process 2026-02-26T03:22:16.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:386: TEST_bluestore2: ceph osd down 0 2026-02-26T03:22:16.676 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:22:16.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:388: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/block --command bluefs-bdev-migrate 2026-02-26T03:22:16.704 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:22:20.337 INFO:tasks.workunit.client.0.trial123.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-26T03:22:20.366 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:393: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command bluefs-bdev-sizes 2026-02-26T03:22:20.385 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:22:21.504 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x100000000(4 GiB) : using 0x3a358000(931 MiB) 2026-02-26T03:22:21.976 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:396: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command fsck 2026-02-26T03:22:24.605 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:22:24.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:399: TEST_bluestore2: activate_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:22:24.693 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:22:24.694 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:22:24.695 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:22:24.696 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:22:24.696 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:22:24.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:22:24.697 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:22:24.697 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-26T03:22:24.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:22:24.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:22:24.698 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:22:24.699 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:22:24.718 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:24.716+0000 7fec77b048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:24.720 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:24.716+0000 7fec77b048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:24.721 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:24.720+0000 7fec77b048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:24.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:22:24.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:22:24.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:22:24.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:22:24.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:22:24.894 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:24.895 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:22:24.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:22:24.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:24.895 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:25.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:25.947 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:25.944+0000 7fec77b048c0 -1 Falling back to public interface 2026-02-26T03:22:26.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:26.102 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:26.103 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:22:26.103 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:22:26.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:26.104 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:26.303 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:27.304 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:27.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:27.305 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:22:27.305 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:22:27.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:27.306 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:27.515 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:28.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:28.516 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:28.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:22:28.517 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:22:28.517 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:28.518 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:28.722 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:29.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:29.723 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:29.724 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:22:29.724 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:22:29.724 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:29.725 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:29.923 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:30.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:30.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:30.924 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:22:30.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:22:30.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:30.924 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:31.117 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:31.764 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:31.760+0000 7fec77b048c0 -1 osd.0 9 log_to_monitors true 2026-02-26T03:22:32.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:32.119 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:32.120 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:22:32.120 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:22:32.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:32.121 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:32.328 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 12 up_thru 12 down_at 10 last_clean_interval [5,9) [v2:127.0.0.1:6802/2589085272,v1:127.0.0.1:6803/2589085272] [v2:127.0.0.1:6804/2589085272,v1:127.0.0.1:6805/2589085272] exists,up 8e6214bf-0a48-43a0-a4b0-284b6c1155a0 2026-02-26T03:22:32.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:22:32.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:22:32.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:22:32.329 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:22:32.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: osd_pid0=144955 2026-02-26T03:22:32.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:402: TEST_bluestore2: wait_for_clean 2026-02-26T03:22:32.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:22:32.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:22:32.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:22:32.331 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:22:32.331 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:22:32.331 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:22:32.331 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:22:32.331 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:22:32.331 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:22:32.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:22:32.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:22:32.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:22:32.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:22:32.426 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:22:32.427 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:22:32.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-26T03:22:32.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:22:32.629 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:22:32.629 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:22:32.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607555 2026-02-26T03:22:32.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607555 2026-02-26T03:22:32.706 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-51539607555' 2026-02-26T03:22:32.707 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:22:32.707 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-51539607555 2026-02-26T03:22:32.707 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:22:32.708 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:22:32.708 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-51539607555 2026-02-26T03:22:32.709 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:22:32.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607555 2026-02-26T03:22:32.710 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 51539607555' 2026-02-26T03:22:32.710 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 51539607555 2026-02-26T03:22:32.711 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:22:32.912 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 51539607555 2026-02-26T03:22:32.912 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:22:33.913 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:22:33.913 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:22:34.114 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607555 -lt 51539607555 2026-02-26T03:22:34.115 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:22:34.115 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:22:34.115 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:22:34.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-26T03:22:34.378 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:22:34.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:22:34.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:22:34.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:22:34.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:22:34.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:22:34.379 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:22:34.579 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-26T03:22:34.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:22:34.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:22:34.580 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:22:34.835 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-26T03:22:34.835 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:22:34.836 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:22:35.146 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:22:35.146 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:22:35.147 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:22:35.147 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:22:35.149 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:22:35.149 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:22:35.150 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:22:35.151 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:22:35.151 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:22:35.152 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:22:35.152 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:22:35.153 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:22:35.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:22:35.154 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-26T03:22:35.320 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:22:35.320 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.321 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.321 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92727 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-26T03:22:35.323 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:22:35.324 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:22:35.325 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:22:35.325 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:22:35.326 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:22:35.326 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:22:35.327 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:22:35.327 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:22:35.328 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:22:35.328 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:22:35.328 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:22:35.329 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:22:35.329 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:22:35.330 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:22:35.331 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-26T03:22:35.331 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-26T03:22:35.332 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:22:35.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.333 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.333 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92727 2026-02-26T03:22:35.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:22:35.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:22:35.334 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-26T03:22:35.336 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-26T03:22:35.336 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.336 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92727 2026-02-26T03:22:35.338 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-26T03:22:35.338 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-26T03:22:35.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-26T03:22:35.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-26T03:22:35.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore_expand td/osd-bluefs-volume-ops 2026-02-26T03:22:35.339 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:406: TEST_bluestore_expand: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.339 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: ulimit -n 2026-02-26T03:22:35.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: local flimit=4096 2026-02-26T03:22:35.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:409: TEST_bluestore_expand: '[' 4096 -lt 1536 ']' 2026-02-26T03:22:35.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: export CEPH_MON=127.0.0.1:7146 2026-02-26T03:22:35.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: CEPH_MON=127.0.0.1:7146 2026-02-26T03:22:35.340 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:413: TEST_bluestore_expand: export CEPH_ARGS 2026-02-26T03:22:35.340 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: uuidgen 2026-02-26T03:22:35.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: CEPH_ARGS+='--fsid=ad91c509-5932-4e06-aee1-6a472d9d630f --auth-supported=none ' 2026-02-26T03:22:35.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:415: TEST_bluestore_expand: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-26T03:22:35.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:416: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-26T03:22:35.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:417: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-26T03:22:35.342 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:418: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:419: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:420: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:421: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:422: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:423: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:425: TEST_bluestore_expand: run_mon td/osd-bluefs-volume-ops a 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-26T03:22:35.343 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.367 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-26T03:22:35.367 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:22:35.367 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:22:35.368 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:22:35.368 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.368 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.368 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:22:35.368 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-26T03:22:35.399 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-26T03:22:35.400 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-26T03:22:35.400 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:22:35.400 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:22:35.400 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-26T03:22:35.400 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-26T03:22:35.400 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:22:35.401 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:22:35.401 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:22:35.401 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:22:35.401 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.401 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.401 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92727/ceph-mon.a.asok 2026-02-26T03:22:35.402 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:22:35.402 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92727/ceph-mon.a.asok config get fsid 2026-02-26T03:22:35.460 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-26T03:22:35.460 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-26T03:22:35.460 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-26T03:22:35.460 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-26T03:22:35.460 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-26T03:22:35.461 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92727/ceph-mon.a.asok 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-26T03:22:35.462 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92727/ceph-mon.a.asok config get mon_host 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:426: TEST_bluestore_expand: run_mgr td/osd-bluefs-volume-ops x 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-26T03:22:35.519 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-26T03:22:35.635 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-26T03:22:35.635 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:22:35.636 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:22:35.636 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:22:35.636 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.636 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.636 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:22:35.637 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:22:35.637 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:427: TEST_bluestore_expand: run_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=ad91c509-5932-4e06-aee1-6a472d9d630f --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-26T03:22:35.661 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:22:35.662 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:22:35.663 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-26T03:22:35.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-26T03:22:35.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:22:35.664 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-26T03:22:35.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3379b323-2acd-45b9-a19f-41ab00734155 2026-02-26T03:22:35.664 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 3379b323-2acd-45b9-a19f-41ab00734155' 2026-02-26T03:22:35.665 INFO:tasks.workunit.client.0.trial123.stdout:add osd0 3379b323-2acd-45b9-a19f-41ab00734155 2026-02-26T03:22:35.665 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-26T03:22:35.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB7vJ9pfOxsKBAAdCKQGr3ALuOyY3sSdRJeNg== 2026-02-26T03:22:35.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB7vJ9pfOxsKBAAdCKQGr3ALuOyY3sSdRJeNg=="}' 2026-02-26T03:22:35.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3379b323-2acd-45b9-a19f-41ab00734155 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-26T03:22:35.790 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:22:35.797 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-26T03:22:35.799 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=ad91c509-5932-4e06-aee1-6a472d9d630f --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB7vJ9pfOxsKBAAdCKQGr3ALuOyY3sSdRJeNg== --osd-uuid 3379b323-2acd-45b9-a19f-41ab00734155 2026-02-26T03:22:35.818 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:35.816+0000 7ff33d7238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:35.819 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:35.816+0000 7ff33d7238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:35.821 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:35.816+0000 7ff33d7238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:35.821 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:35.816+0000 7ff33d7238c0 -1 bdev(0x5611e739ac00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-26T03:22:35.821 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:35.816+0000 7ff33d7238c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-26T03:22:40.718 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-26T03:22:40.719 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-26T03:22:40.720 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-26T03:22:40.720 INFO:tasks.workunit.client.0.trial123.stdout:adding osd0 key to auth repository 2026-02-26T03:22:40.721 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-26T03:22:41.002 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-26T03:22:41.002 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:22:41.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=ad91c509-5932-4e06-aee1-6a472d9d630f --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:22:41.003 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-26T03:22:41.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-26T03:22:41.004 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-26T03:22:41.032 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:41.028+0000 7f5ec36c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:41.034 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:41.032+0000 7f5ec36c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:41.034 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:41.032+0000 7f5ec36c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:22:41.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-26T03:22:41.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:22:41.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:22:41.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:22:41.199 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:22:41.200 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:22:41.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:41.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:22:41.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:41.200 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:41.402 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:42.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:42.404 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:42.404 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:22:42.405 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:22:42.405 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:42.405 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:42.608 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:43.011 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:43.008+0000 7f5ec36c68c0 -1 Falling back to public interface 2026-02-26T03:22:43.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:43.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:43.609 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:22:43.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:22:43.609 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:43.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:43.820 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:44.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:44.822 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:44.822 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:22:44.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:22:44.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:44.823 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:45.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:46.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:46.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:46.029 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:22:46.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:22:46.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:46.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:46.230 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:47.231 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:47.232 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:47.232 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:22:47.233 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:22:47.233 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:47.233 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:47.438 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:22:47.559 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:22:47.556+0000 7f5ec36c68c0 -1 osd.0 0 log_to_monitors true 2026-02-26T03:22:48.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:22:48.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:22:48.439 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:22:48.440 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:22:48.440 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:22:48.440 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:22:48.644 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/98177124,v1:127.0.0.1:6803/98177124] [v2:127.0.0.1:6804/98177124,v1:127.0.0.1:6805/98177124] exists,up 3379b323-2acd-45b9-a19f-41ab00734155 2026-02-26T03:22:48.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:22:48.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:22:48.645 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:22:48.645 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:22:48.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: osd_pid0=147691 2026-02-26T03:22:48.646 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:430: TEST_bluestore_expand: sleep 5 2026-02-26T03:22:53.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:431: TEST_bluestore_expand: create_pool foo 16 2026-02-26T03:22:53.647 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-26T03:22:53.897 INFO:tasks.workunit.client.0.trial123.stderr:pool 'foo' already exists 2026-02-26T03:22:53.906 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-26T03:22:54.908 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:434: TEST_bluestore_expand: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-26T03:22:54.932 INFO:tasks.workunit.client.0.trial123.stdout:hints = 1 2026-02-26T03:22:54.932 INFO:tasks.workunit.client.0.trial123.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-26T03:22:54.932 INFO:tasks.workunit.client.0.trial123.stdout:Object prefix: benchmark_data_trial123_150035 2026-02-26T03:22:54.932 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:22:54.932 INFO:tasks.workunit.client.0.trial123.stdout: 0 0 0 0 0 0 - 0 2026-02-26T03:22:55.932 INFO:tasks.workunit.client.0.trial123.stdout: 1 16 13832 13816 53.9644 53.9688 0.00106105 0.0011565 2026-02-26T03:22:56.932 INFO:tasks.workunit.client.0.trial123.stdout: 2 16 27619 27603 53.9079 53.8555 0.000909927 0.00115804 2026-02-26T03:22:57.932 INFO:tasks.workunit.client.0.trial123.stdout: 3 16 41538 41522 54.061 54.3711 0.00112668 0.00115499 2026-02-26T03:22:58.932 INFO:tasks.workunit.client.0.trial123.stdout: 4 16 55323 55307 54.0067 53.8477 0.000997589 0.00115615 2026-02-26T03:22:59.932 INFO:tasks.workunit.client.0.trial123.stdout: 5 16 69078 69062 53.9506 53.7305 0.00125488 0.00115735 2026-02-26T03:23:00.933 INFO:tasks.workunit.client.0.trial123.stdout: 6 16 82894 82878 53.953 53.9688 0.00130408 0.00115739 2026-02-26T03:23:01.933 INFO:tasks.workunit.client.0.trial123.stdout: 7 16 96681 96665 53.9384 53.8555 0.00138265 0.00115768 2026-02-26T03:23:02.933 INFO:tasks.workunit.client.0.trial123.stdout: 8 15 109903 109888 53.6522 51.6523 0.00145229 0.00116388 2026-02-26T03:23:03.933 INFO:tasks.workunit.client.0.trial123.stdout: 9 16 122156 122140 53.0081 47.8594 0.00127798 0.00117806 2026-02-26T03:23:04.933 INFO:tasks.workunit.client.0.trial123.stdout: 10 16 134606 134590 52.5702 48.6328 0.00137495 0.00118786 2026-02-26T03:23:05.933 INFO:tasks.workunit.client.0.trial123.stdout: 11 16 146749 146733 52.1029 47.4336 0.00147638 0.00119855 2026-02-26T03:23:06.933 INFO:tasks.workunit.client.0.trial123.stdout: 12 16 158846 158830 51.6985 47.2539 0.00103632 0.00120792 2026-02-26T03:23:07.933 INFO:tasks.workunit.client.0.trial123.stdout: 13 16 170515 170499 51.2277 45.582 0.00152486 0.00121904 2026-02-26T03:23:08.933 INFO:tasks.workunit.client.0.trial123.stdout: 14 16 170980 170964 47.6984 1.81641 0.0229722 0.00130858 2026-02-26T03:23:09.933 INFO:tasks.workunit.client.0.trial123.stdout: 15 16 171396 171380 44.6268 1.625 0.199417 0.00139225 2026-02-26T03:23:10.933 INFO:tasks.workunit.client.0.trial123.stdout: 16 16 174455 174439 42.5844 11.9492 0.00136772 0.00146667 2026-02-26T03:23:11.933 INFO:tasks.workunit.client.0.trial123.stdout: 17 16 186139 186123 42.7639 45.6406 0.00134935 0.0014605 2026-02-26T03:23:12.933 INFO:tasks.workunit.client.0.trial123.stdout: 18 15 197714 197699 42.9001 45.2188 0.00131884 0.00145588 2026-02-26T03:23:13.933 INFO:tasks.workunit.client.0.trial123.stdout: 19 16 209214 209198 43.0061 44.918 0.00215694 0.00145228 2026-02-26T03:23:14.933 INFO:tasks.workunit.client.0.trial123.stdout:2026-02-26T03:23:14.936685+0000 min lat: 0.000472696 max lat: 0.256393 avg lat: 0.00144895 2026-02-26T03:23:14.933 INFO:tasks.workunit.client.0.trial123.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-26T03:23:14.933 INFO:tasks.workunit.client.0.trial123.stdout: 20 16 220729 220713 43.1047 44.9805 0.00152272 0.00144895 2026-02-26T03:23:15.934 INFO:tasks.workunit.client.0.trial123.stdout: 21 16 232185 232169 43.1828 44.75 0.00134604 0.00144634 2026-02-26T03:23:16.934 INFO:tasks.workunit.client.0.trial123.stdout: 22 16 243627 243611 43.2514 44.6953 0.00111547 0.00144404 2026-02-26T03:23:17.934 INFO:tasks.workunit.client.0.trial123.stdout: 23 16 255179 255163 43.3327 45.125 0.00150669 0.00144134 2026-02-26T03:23:18.934 INFO:tasks.workunit.client.0.trial123.stdout: 24 16 267036 267020 43.4569 46.3164 0.00143477 0.00143722 2026-02-26T03:23:19.934 INFO:tasks.workunit.client.0.trial123.stdout: 25 16 278637 278621 43.5312 45.3164 0.00113034 0.00143476 2026-02-26T03:23:20.934 INFO:tasks.workunit.client.0.trial123.stdout: 26 16 290282 290266 43.6063 45.4883 0.00104034 0.00143229 2026-02-26T03:23:21.934 INFO:tasks.workunit.client.0.trial123.stdout: 27 16 301962 301946 43.6809 45.625 0.00115151 0.00142985 2026-02-26T03:23:22.934 INFO:tasks.workunit.client.0.trial123.stdout: 28 16 313537 313521 43.7356 45.2148 0.00195272 0.00142806 2026-02-26T03:23:23.934 INFO:tasks.workunit.client.0.trial123.stdout: 29 16 325303 325287 43.8122 45.9609 0.00141134 0.00142556 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Total time run: 30.0013 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Total writes made: 338110 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Write size: 4096 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Object size: 4096 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Bandwidth (MB/sec): 44.0229 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Bandwidth: 13.9373 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Max bandwidth (MB/sec): 54.3711 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Min bandwidth (MB/sec): 1.625 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Average IOPS: 11269 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Stddev IOPS: 3567.95 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Max IOPS: 13919 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Min IOPS: 416 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Average Latency(s): 0.00141875 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Stddev Latency(s): 0.00332131 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Max latency(s): 0.256393 2026-02-26T03:23:24.935 INFO:tasks.workunit.client.0.trial123.stdout:Min latency(s): 0.000472696 2026-02-26T03:23:24.937 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:435: TEST_bluestore_expand: sleep 5 2026-02-26T03:23:29.939 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-26T03:23:29.940 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-26T03:23:30.019 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: total_space_before=4294967296 2026-02-26T03:23:30.020 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-26T03:23:30.020 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-26T03:23:30.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep free 2026-02-26T03:23:30.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d: -f 2 2026-02-26T03:23:30.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d, -f 1 2026-02-26T03:23:30.021 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-26T03:23:30.100 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: free_space_before=2909970432 2026-02-26T03:23:30.100 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 147691 2026-02-26T03:23:30.100 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-26T03:23:31.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 147691 2026-02-26T03:23:31.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 441: kill: (147691) - No such process 2026-02-26T03:23:31.101 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:442: TEST_bluestore_expand: ceph osd down 0 2026-02-26T03:23:31.300 INFO:tasks.workunit.client.0.trial123.stderr:osd.0 is already down. 2026-02-26T03:23:31.310 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:445: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 allocmap 2026-02-26T03:23:31.336 INFO:tasks.workunit.client.0.trial123.stdout:allocmap bluestore.allocmap 2026-02-26T03:24:12.444 INFO:tasks.workunit.client.0.trial123.stdout:allocmap success 2026-02-26T03:24:12.492 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:448: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:24:56.865 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:24:56.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:450: TEST_bluestore_expand: requested_space=4294967296 2026-02-26T03:24:56.915 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:451: TEST_bluestore_expand: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-26T03:24:56.916 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:452: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-26T03:24:56.938 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:25:36.060 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0x28100000(641 MiB) 2026-02-26T03:25:36.060 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0x100000000(4 GiB) : using 0x528d8000(1.3 GiB) 2026-02-26T03:25:36.060 INFO:tasks.workunit.client.0.trial123.stdout:Expanding DB/WAL... 2026-02-26T03:25:36.060 INFO:tasks.workunit.client.0.trial123.stdout:1 : nothing to do, skipped 2026-02-26T03:25:36.060 INFO:tasks.workunit.client.0.trial123.stdout:2 : nothing to do, skipped 2026-02-26T03:25:36.805 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:455: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-26T03:26:20.922 INFO:tasks.workunit.client.0.trial123.stdout:fsck success 2026-02-26T03:26:20.970 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:458: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 qfsck 2026-02-26T03:26:20.988 INFO:tasks.workunit.client.0.trial123.stdout:qfsck bluestore.quick-fsck 2026-02-26T03:27:02.090 INFO:tasks.workunit.client.0.trial123.stdout:qfsck success 2026-02-26T03:27:02.135 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:460: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-26T03:27:02.156 INFO:tasks.workunit.client.0.trial123.stdout:inferring bluefs devices from bluestore path 2026-02-26T03:27:42.296 INFO:tasks.workunit.client.0.trial123.stdout:1 : device size 0x40000000(1 GiB) : using 0x28100000(641 MiB) 2026-02-26T03:27:42.296 INFO:tasks.workunit.client.0.trial123.stdout:2 : device size 0x100000000(4 GiB) : using 0x528d8000(1.3 GiB) 2026-02-26T03:27:43.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:462: TEST_bluestore_expand: activate_osd td/osd-bluefs-volume-ops 0 2026-02-26T03:27:43.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:27:43.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-26T03:27:43.024 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=ad91c509-5932-4e06-aee1-6a472d9d630f --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-26T03:27:43.025 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:27:43.026 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-26T03:27:43.027 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-26T03:27:43.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-26T03:27:43.028 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-26T03:27:43.028 INFO:tasks.workunit.client.0.trial123.stdout:start osd.0 2026-02-26T03:27:43.029 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-26T03:27:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=8c737ca3-f5bc-4dce-827e-c4b6a0b1ec4b --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=f7f05045-da16-4bdd-938e-aae18b73a3ef --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=ad91c509-5932-4e06-aee1-6a472d9d630f --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92727/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-26T03:27:43.030 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-26T03:27:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-26T03:27:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-26T03:27:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-26T03:27:43.030 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-26T03:27:43.051 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:27:43.048+0000 7f63e31f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:27:43.052 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:27:43.048+0000 7f63e31f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:27:43.053 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:27:43.048+0000 7f63e31f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:43.235 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-26T03:27:43.236 INFO:tasks.workunit.client.0.trial123.stdout:0 2026-02-26T03:27:43.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:43.236 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:43.436 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:44.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:44.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:44.437 INFO:tasks.workunit.client.0.trial123.stdout:1 2026-02-26T03:27:44.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-26T03:27:44.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:44.437 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:44.567 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:27:44.564+0000 7f63e31f58c0 -1 Falling back to public interface 2026-02-26T03:27:44.639 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:45.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:45.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:45.640 INFO:tasks.workunit.client.0.trial123.stdout:2 2026-02-26T03:27:45.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-26T03:27:45.640 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:45.641 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:45.841 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:46.842 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:46.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:46.843 INFO:tasks.workunit.client.0.trial123.stdout:3 2026-02-26T03:27:46.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-26T03:27:46.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:46.843 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:47.051 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:48.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:48.053 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:48.053 INFO:tasks.workunit.client.0.trial123.stdout:4 2026-02-26T03:27:48.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-26T03:27:48.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:48.054 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:48.260 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:49.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:49.261 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:49.262 INFO:tasks.workunit.client.0.trial123.stdout:5 2026-02-26T03:27:49.262 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-26T03:27:49.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:49.263 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:49.470 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:50.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:50.471 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:50.472 INFO:tasks.workunit.client.0.trial123.stdout:6 2026-02-26T03:27:50.472 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-26T03:27:50.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:50.473 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:50.675 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:51.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:51.676 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:51.677 INFO:tasks.workunit.client.0.trial123.stdout:7 2026-02-26T03:27:51.677 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-26T03:27:51.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:51.678 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:51.879 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:52.880 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:52.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:52.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-26T03:27:52.881 INFO:tasks.workunit.client.0.trial123.stdout:8 2026-02-26T03:27:52.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:52.882 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:53.087 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:54.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:54.088 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:54.089 INFO:tasks.workunit.client.0.trial123.stdout:9 2026-02-26T03:27:54.089 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-26T03:27:54.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:54.090 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:54.756 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:55.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:55.758 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:55.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-26T03:27:55.759 INFO:tasks.workunit.client.0.trial123.stdout:10 2026-02-26T03:27:55.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:55.759 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:55.967 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:56.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:56.968 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:56.969 INFO:tasks.workunit.client.0.trial123.stdout:11 2026-02-26T03:27:56.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-26T03:27:56.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:56.969 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:57.173 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:58.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:58.174 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:58.175 INFO:tasks.workunit.client.0.trial123.stdout:12 2026-02-26T03:27:58.175 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 12 2026-02-26T03:27:58.175 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:58.175 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:58.381 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-26T03:27:59.106 INFO:tasks.workunit.client.0.trial123.stderr:2026-02-26T03:27:59.107+0000 7f63e31f58c0 -1 osd.0 10 log_to_monitors true 2026-02-26T03:27:59.382 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-26T03:27:59.383 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-26T03:27:59.383 INFO:tasks.workunit.client.0.trial123.stdout:13 2026-02-26T03:27:59.384 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 13 2026-02-26T03:27:59.384 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-26T03:27:59.384 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-26T03:27:59.581 INFO:tasks.workunit.client.0.trial123.stdout:osd.0 up in weight 1 up_from 13 up_thru 13 down_at 10 last_clean_interval [5,10) [v2:127.0.0.1:6802/685463907,v1:127.0.0.1:6803/685463907] [v2:127.0.0.1:6804/685463907,v1:127.0.0.1:6805/685463907] exists,up 3379b323-2acd-45b9-a19f-41ab00734155 2026-02-26T03:27:59.581 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-26T03:27:59.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-26T03:27:59.582 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-26T03:27:59.582 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-26T03:27:59.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: osd_pid0=152401 2026-02-26T03:27:59.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:465: TEST_bluestore_expand: wait_for_clean 2026-02-26T03:27:59.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-26T03:27:59.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-26T03:27:59.583 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-26T03:27:59.583 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-26T03:27:59.584 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-26T03:27:59.584 INFO:tasks.workunit.client.0.trial123.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-26T03:27:59.584 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-26T03:27:59.584 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-26T03:27:59.585 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-26T03:27:59.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-26T03:27:59.684 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-26T03:27:59.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-26T03:27:59.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-26T03:27:59.685 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-26T03:27:59.685 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-26T03:27:59.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-26T03:27:59.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-26T03:27:59.899 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-26T03:27:59.900 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-26T03:27:59.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-26T03:27:59.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-26T03:27:59.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-55834574851' 2026-02-26T03:27:59.982 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-26T03:27:59.983 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-55834574851 2026-02-26T03:27:59.983 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-26T03:27:59.984 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-26T03:27:59.985 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-55834574851 2026-02-26T03:27:59.985 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-26T03:27:59.986 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-26T03:27:59.986 INFO:tasks.workunit.client.0.trial123.stdout:waiting osd.0 seq 55834574851 2026-02-26T03:27:59.987 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 55834574851' 2026-02-26T03:27:59.987 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:28:00.193 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 55834574851 2026-02-26T03:28:00.194 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:28:01.195 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-26T03:28:01.195 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:28:01.396 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 55834574851 2026-02-26T03:28:01.396 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-26T03:28:02.396 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-26T03:28:02.397 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-26T03:28:02.610 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-26T03:28:02.611 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-26T03:28:02.611 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:28:02.611 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:28:02.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-26T03:28:02.881 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-26T03:28:02.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-26T03:28:02.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-26T03:28:02.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-26T03:28:02.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-26T03:28:02.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-26T03:28:02.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-26T03:28:03.078 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-26T03:28:03.079 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-26T03:28:03.079 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-26T03:28:03.079 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-26T03:28:03.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-26T03:28:03.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-26T03:28:03.336 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-26T03:28:03.337 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-26T03:28:03.337 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-26T03:28:03.419 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: total_space_after=4294967296 2026-02-26T03:28:03.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-26T03:28:03.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-26T03:28:03.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep free 2026-02-26T03:28:03.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d: -f 2 2026-02-26T03:28:03.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d, -f 1 2026-02-26T03:28:03.421 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-26T03:28:03.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: free_space_after=2900221952 2026-02-26T03:28:03.497 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:470: TEST_bluestore_expand: '[' 4294967296 '!=' 4294967296 ']' 2026-02-26T03:28:03.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:476: TEST_bluestore_expand: total_space_added=0 2026-02-26T03:28:03.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:477: TEST_bluestore_expand: free_space_added=-9748480 2026-02-26T03:28:03.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:479: TEST_bluestore_expand: let 'new_used_space=(0 - -9748480)' 2026-02-26T03:28:03.498 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:480: TEST_bluestore_expand: echo 9748480 2026-02-26T03:28:03.498 INFO:tasks.workunit.client.0.trial123.stdout:9748480 2026-02-26T03:28:03.499 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:482: TEST_bluestore_expand: '[' 9748480 -gt 131072 ']' 2026-02-26T03:28:03.499 INFO:tasks.workunit.client.0.trial123.stdout:total_space_added = 0 2026-02-26T03:28:03.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:483: TEST_bluestore_expand: echo 'total_space_added = 0' 2026-02-26T03:28:03.500 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:484: TEST_bluestore_expand: echo 'free_space_added = -9748480' 2026-02-26T03:28:03.500 INFO:tasks.workunit.client.0.trial123.stdout:free_space_added = -9748480 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:485: TEST_bluestore_expand: return 1 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: return 1 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2379: main: code=1 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-bluefs-volume-ops 1 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=1 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-26T03:28:03.502 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-26T03:28:03.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-26T03:28:03.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-26T03:28:03.503 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-26T03:28:03.807 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-26T03:28:03.807 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-26T03:28:03.808 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-26T03:28:03.809 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-26T03:28:03.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-26T03:28:03.810 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-26T03:28:03.811 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-26T03:28:03.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:28:03.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-26T03:28:03.812 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-26T03:28:03.813 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-26T03:28:03.814 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:28:03.815 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-26T03:28:03.815 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: '[' -n '' ']' 2026-02-26T03:28:03.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:194: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-26T03:28:03.816 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:195: teardown: mv td/osd-bluefs-volume-ops/bluestore_tool.log td/osd-bluefs-volume-ops/mgr.x.log td/osd-bluefs-volume-ops/mon.a.log td/osd-bluefs-volume-ops/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-26T03:28:03.818 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-26T03:28:03.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-26T03:28:03.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-26T03:28:03.882 INFO:tasks.workunit.client.0.trial123.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92727 2026-02-26T03:28:03.883 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92727 2026-02-26T03:28:03.883 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-26T03:28:03.884 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-26T03:28:03.884 INFO:tasks.workunit.client.0.trial123.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 1 2026-02-26T03:28:03.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:28:03.885 INFO:tasks.workunit:Stopping ['osd'] on client.0... 2026-02-26T03:28:03.885 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-26T03:28:04.187 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh' 2026-02-26T03:28:04.193 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-26T03:28:04.209 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-26T03:28:04.209 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-26T03:28:04.318 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-26T03:28:04.330 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-26T03:28:04.330 DEBUG:teuthology.orchestra.run.trial123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-26T03:28:04.396 INFO:teuthology.orchestra.run.trial123.stdout: remote refid st t when poll reach delay offset jitter 2026-02-26T03:28:04.396 INFO:teuthology.orchestra.run.trial123.stdout:============================================================================== 2026-02-26T03:28:04.396 INFO:teuthology.orchestra.run.trial123.stdout:-172-233-157-223 82.35.162.146 2 u 33 64 377 66.533 +13.716 1.480 2026-02-26T03:28:04.396 INFO:teuthology.orchestra.run.trial123.stdout:*209.177.158.85 128.233.154.245 2 u 41 64 377 19.926 -3.835 0.675 2026-02-26T03:28:04.396 INFO:teuthology.orchestra.run.trial123.stdout:+srcf-ntp.stanfo 171.64.7.105 2 u 42 64 377 67.872 -4.451 0.953 2026-02-26T03:28:04.396 INFO:teuthology.orchestra.run.trial123.stdout:+ntp.speculation 128.9.176.30 2 u 51 64 377 61.990 -1.803 2.767 2026-02-26T03:28:04.397 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-26T03:28:04.408 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-26T03:28:04.409 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-26T03:28:04.421 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-26T03:28:04.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-26T03:28:04.450 INFO:teuthology.task.internal:Duration was 1957.361303 seconds 2026-02-26T03:28:04.451 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-26T03:28:04.467 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-26T03:28:04.468 DEBUG:teuthology.orchestra.run.trial123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-26T03:28:04.493 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-26T03:28:04.494 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial123.front.sepia.ceph.com 2026-02-26T03:28:04.494 DEBUG:teuthology.orchestra.run.trial123:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-26T03:28:04.543 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-26T03:28:04.544 DEBUG:teuthology.orchestra.run.trial123:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:28:04.810 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-26T03:28:04.810 DEBUG:teuthology.orchestra.run.trial123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-26T03:28:04.820 INFO:teuthology.orchestra.run.trial123.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-26T03:28:04.821 INFO:teuthology.orchestra.run.trial123.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-26T03:28:04.821 INFO:teuthology.orchestra.run.trial123.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-26T03:28:04.822 INFO:teuthology.orchestra.run.trial123.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-26T03:28:04.822 INFO:teuthology.orchestra.run.trial123.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-26T03:28:04.891 INFO:teuthology.orchestra.run.trial123.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-26T03:28:04.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-26T03:28:04.909 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-26T03:28:04.910 DEBUG:teuthology.orchestra.run.trial123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-26T03:28:04.942 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-26T03:28:04.961 DEBUG:teuthology.orchestra.run.trial123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:28:04.993 INFO:teuthology.orchestra.run.trial123.stdout:kernel.core_pattern = core 2026-02-26T03:28:05.003 DEBUG:teuthology.orchestra.run.trial123:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-26T03:28:05.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T03:28:05.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-26T03:28:05.065 INFO:teuthology.task.internal:Transferring archived files... 2026-02-26T03:28:05.069 DEBUG:teuthology.misc:Transferring archived files from trial123:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-26_01:51:16-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/71112/remote/trial123 2026-02-26T03:28:05.070 DEBUG:teuthology.orchestra.run.trial123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-26T05:06:04.565 INFO:teuthology.task.internal:Removing archive directory... 2026-02-26T05:06:04.572 DEBUG:teuthology.orchestra.run.trial123:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-26T05:06:07.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-26T05:06:07.720 INFO:teuthology.task.internal:Not uploading archives. 2026-02-26T05:06:07.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-26T05:06:07.736 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-26T05:06:07.737 DEBUG:teuthology.orchestra.run.trial123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-26T05:06:07.742 INFO:teuthology.orchestra.run.trial123.stdout: 1311292 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 26 05:06 /home/ubuntu/cephtest 2026-02-26T05:06:07.742 INFO:teuthology.orchestra.run.trial123.stdout: 1320633 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 26 02:58 /home/ubuntu/cephtest/mnt.0 2026-02-26T05:06:07.742 INFO:teuthology.orchestra.run.trial123.stdout: 1320634 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 26 03:11 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-26T05:06:07.742 INFO:teuthology.orchestra.run.trial123.stdout: 1335425 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 26 03:11 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-26T05:06:07.743 INFO:teuthology.orchestra.run.trial123.stdout: 1335426 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 26 03:28 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-26T05:06:07.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-26T05:06:07.744 INFO:teuthology.orchestra.run.trial123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-26T05:06:07.744 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial123 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-26T05:06:07.748 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-26T05:06:07.764 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-26T05:06:07.781 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial123 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-26T05:06:07.786 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} duration: 1957.3613033294678 failure_reason: 'Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial123 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8d724284d6d767af9ab8bf4bc6bd4e45f8502cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-26T05:06:07.787 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-26T05:06:07.908 INFO:teuthology.run:FAIL